Building a social app with Spin (1/4): Project Setup and first API
Justin Pflueger
social
spin
wasm
Hello Fermyon friends! I’m Justin Pflueger and I’m a Senior Solutions Engineer here at Fermyon. I’d like to share a project with you that I’ve been working on - and take you through the process of creating and shipping an app using WebAssembly, step-by-step. But I’m a sucker for context and I like to dig into the who, why and what to better understand with the how. However, we’re all busy so you can always just skip straight to the What or How sections below if you want.
Who
You can’t trust just any random stranger on the internet so let me tell you about myself a little bit. Like many of you I’ve been a software developer for a while now, writing mostly closed source applications for various companies during my career. A lot of my experience has been in Java, C#, JavaScript and general web development but I’ve dabbled in Python, Golang, Scala and a few others.
Before joining Fermyon WebAssembly was just a buzzword to me; never really using it in a meaningful project before because it wasn’t stable or widely used enough for it to be worth the effort of ramping up when there are project deadlines to consider. I’ve learned a ton since joining Fermyon but I’m still very much a newcomer to world of WebAssembly. Likewise, I’ve never written a blog post or anything really that wasn’t a README or architectural design, so bear with me and please let me know what you think of the project/article on GitHub or in our Fermyon Discord Server.
Why
Now let’s talk about Fermyon. If you’ve made it this far without falling asleep chances are you already know who Fermyon is and how we’re pioneering the next wave of cloud computing. Our primary goal this year is to empower developers to build real-world applications using the open-source Spin and the Fermyon Platform projects or with Spin and Fermyon Cloud.
There are already great examples out there of building fun applications like Finicky Whiskers or writing web hooks with Spin. However awesome, these apps aren’t what you or I would develop in our day-to-day work. To pull off our ambitious goal this year, we feel strongly that we needed to put ourselves in the shoes of our developers by building a real-world application ourselves. We’re missing out on patterns like authentication, authorization, CRUD micro-services, CICD, etc.
This article is just the beginning of a journey. Along this journey, we’re going to build a real-world application to exercise these patterns. I’m going to be critical and open about my developer experience with the intention of improving our ecosystem through feedback to our product and engineering groups. I’m going to make mistakes, find bugs, create workarounds and suggest improvements. As I said, this is just the beginning and I’d love for you to join me on this journey. You can follow along however you’d like, skip straight to the code (beware it’s still WIP), read my future blog posts or even collaborate on the project directly on GitHub.
What
The Fermyon team discussed a few different domains for the kind of application we should build: e-commerce, B2B integrations, and social media, to name a few. We felt that building a social media application would give us the breadth of features more typical of a real-world application.
We could just build a clone of an existing social media app, but I think we should have a little fun with the specifics. My idea was to have the social aspect center around sharing and discussing code snippets. For the MVP, I want to support sharing a permalink to a range of code in a public repository and rendering that inside of the post. Here’s an example from a GitHub permalink:
https://github.com/fermyon/spin/blob/eb3de4f072f4dfe964d4e2eb564e02b8e1faa012/examples/rust-outbound-mysql/src/model.rs#L15-L27
That link would render this snippet inside of the post:
pub(crate) fn as_pet(row: &mysql::Row) -> Result<Pet> {
let id = i32::decode(&row[0])?;
let name = String::decode(&row[1])?;
let prey = Option::<String>::decode(&row[2])?;
let is_finicky = bool::decode(&row[3])?;
Ok(Pet {
id,
name,
prey,
is_finicky,
})
}
But to be a real-world application, we’re going to need to go further than just POST-ing and GET-ing some JSON. Let’s expand the scope to include some essential features like creating a user profile, authorization on API endpoints, persisting to a database and automating deployments. If you can think of other features, feel free to add an issue to the GitHub project and if time allows I’ll take a stab at implementing it. If you wear a PM hat sometimes and want to see the roadmap of features, check out the planning project in GitHub.
To be clear, I don’t have all of the code written. In this article, we’re going to focus on setting up a Spin project with a module for the single page web app and another module to allow us to create a user profile. We’ll get to authorization and other in-depth topics in future blog posts, so stay tuned.
How
Finally, my context itch has been scratched and we can get to what every developer prefers to spend their time on: writing code.
Decisions Decisions
To get started, we’ll need to decide on a few technologies. We’ll at least need a database and front-end app. I chose MySQL for the database based on popularity and Vue.js for the front-end based both on popularity and because I’m curious about it. Additionally, we will need to pick a programming language for our first REST API endpoint.
Since we’re using WebAssembly, we want to test the path of using multiple languages to embrace the scenario where different development teams may have different preferences and needs based on their owned domains. One of the common selling points of WebAssembly is to write in whatever language you want to. So let’s put that to the test. Since we’re in the world of WebAssembly I’ve chosen to write the profile service in Rust. With one GIGANTIC caveat, I’d never written Rust before joining Fermyon. Luckily, I’ll have some help with this one.
Folder Structure
Ideally the folder structure matches the API routing as closely as possible, aside from the static web app which will assume the root HTTP route. This is how I anticipate we’ll lay out our code to support multiple components in our Spin application to start:
.
├── ...
├── spin.toml
└── web
├── package.json
└── api
└── profile
├── Cargo.toml
Init: Profile REST API
Let’s start out by creating our first REST API for the user’s profile. We want to support basic CRUD operations. Let’s keep it simple at first and define our Profile data model using a sample in JSON format:
{
"id": "1F26C280-94A7-4561-991D-08F48187C83D",
"username": "justin",
"avatar": "https://avatars.githubusercontent.com/u/3060890?v=4"
}
Quick summary of the fields:
| Field | Purpose |
|---|
| id | Unique identifier for the user’s profile. The sample is a UUID but until we know more, we’ll let it be any kind of string |
| username | Public username for the user that will be displayed with their posts |
| avatar | Any URL that we can use to display a profile picture of the user, basic social media site stuff. |
Ready to stub out the first Spin component! Note the /... suffix on the HTTP path which is a wildcard, so anything matching the beginning of the route will be handled by this Rust component.
❯ spin new --output api/profile http-rust profile-api
Project description: Social media app for code snippets
HTTP base: /
HTTP path: /api/profile/...
The output argument will create the spin app in the api/profile directory using the http-rust template that comes installed with the Spin SDK. The final parameter is the name that’s used both in spin.toml and Cargo.toml. Now I want to move the spin.toml file to the repository root and make use of the component.build.workdir config so I can run spin build from the repository’s root.
❯ mv api/profile/spin.toml .
We also need to adjust the component.source and add the working directory config.
Let’s take a peek at the diff so you can see what I’m talking about. Quick diff of the required changes:
❯ git diff HEAD^ -M -- spin.toml api/profile/spin.toml
diff --git a/api/profile/spin.toml b/spin.toml
similarity index 75%
rename from api/profile/spin.toml
rename to spin.toml
index 56b54eb..6f51764 100644
--- a/api/profile/spin.toml
+++ b/spin.toml
@@ -1,14 +1,15 @@
spin_version = "1"
authors = ["Justin Pflueger <justin.pflueger@fermyon.com>"]
description = "Social media app for code snippets"
-name = "profile-api"
+name = "code-things"
trigger = { type = "http", base = "/" }
version = "0.1.0"
[[component]]
id = "profile-api"
-source = "target/wasm32-wasi/release/profile_api.wasm"
+source = "api/profile/target/wasm32-wasi/release/profile_api.wasm"
[component.trigger]
route = "/api/profile/..."
[component.build]
command = "cargo build --target wasm32-wasi --release"
+workdir = "api/profile"
Testing out the endpoint yields the expected Hello, Fermyon and we can see the headers printed in the console. This handy command lets us re-run the code, including any new changes.
❯ spin build --up
Successfully ran the build command for the Spin components.
Serving http://127.0.0.1:3000
Available Routes:
profile-api: http://127.0.0.1:3000/api/profile (wildcard)
{"host": "127.0.0.1:3000", "upgrade-insecure-requests": "1", "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15", "accept-language": "en-US,en;q=0.9", "accept-encoding": "gzip, deflate", "connection": "keep-alive", "spin-path-info": "", "spin-full-url": "http://127.0.0.1:3000/api/profile", "spin-matched-route": "/api/profile/...", "spin-base-path": "/", "spin-raw-component-route": "/api/profile/...", "spin-component-route": "/api/profile"}
One of my most-used shortcuts when writing a spin app has been building and running the app in the same build command. The --up flag will automagically run the spin up command, passing every argument after it to the command.
Setup the Database
Now since I’m a Rust newbie and don’t want to submit you to me stumbling through figuring out patterns, I’m going to follow some patterns from others more knowledgable than myself. Luckily for me, Thorsten Hans posted about my exact use-case already in his post CRUD in WebAssembly with Fermyon Spin and MySQL. I highly recommend taking a moment to read Thorsten’s article and check out the GitHub repository for the article.
I won’t bore you with the database setup, suffice to say I am using docker to run a mysql instance. There’s a script in the repository under scripts/db/up.sh that starts the container, then I used a database client to run the following creation script:
CREATE TABLE `profiles` (
`id` varchar(64) NOT NULL,
`handle` varchar(32) NOT NULL,
`avatar` varchar(256),
UNIQUE(`handle`),
PRIMARY KEY (`id`)
);
Next up is to wire up the database to our application. After reading up on our config docs, I’m going to try and use variables to individually substitute values for the database user, password, host and database name. Eventually we will want to replace this with a secret but for now, we’ll just use variables to supply the configuration to the API. More changes to the spin.toml:
...
[variables]
db_user = { default = "code-things" }
db_pswd = { default = "password" }
db_host = { default = "127.0.0.1" }
db_name = { default = "code_things" }
...
[component.config]
db_url = "mysql://{{db_user}}:{{db_pswd}}@{{db_host}}/{{db_name}}"
Getting the config value in the app proved fairly simple, I wrapped it up into a struct in case we need more config values later:
use spin_sdk::config::get;
const KEY_DB_URL: &str = "db_url";
#[derive(Debug)]
pub(crate) struct Config {
pub db_url: String,
}
impl Config {
pub(crate) fn get<'a>() -> Config {
Config {
db_url: get(KEY_DB_URL).unwrap(),
}
}
}
I verified by logging the url to stdout so I could make sure the changes were working. After years of being spoiled by attached debuggers, it would be nice if there was more obvious support for debugging running code. It does take me back to debugging via stdout though, good memories 🙂.
This is where I have adapted Thorsten’s CRUD sample to utilize a different model for our profile. The Rust code for the Profile API in GitHub should be strikingly similar to Thorsten’s code in GitHub. It’s a really thorough write-up of Spin CRUD applications and he does a much better job of explaining it. There is a significant amount of code to do this and it would be difficult to convey in this blog post but I’m going to attempt to summarize by file but if you can also view the whole commit by clicking the following link:
This file contains the bulk of the API routing and handlers for each of the API’s actions. The profile_api function is responsible for mapping from API actions to the individual handlers: handle_create, handle_read_by_handle, handle_update, handle_delete_by_handle. The api_from_request function is responsible for mapping from a Rust HTTP Request to an API action. Credit to Thorsten Hans for this pattern, it’s largely the same as his version but I did try to simplify a few things here for readability.
The Profile struct in this file is our data model for this API. I’ve also implemented a few helper functions in this file to perform the database operations: insert, get_by_handle, update, and delete.
I really liked how clean the handler logic gets after a few tweaks to Thorsten’s CRUD pattern but this is really just Rust code, so let’s dig into the MySQL support in Spin’s Rust SDK. Let’s take a quick peek at how we interact with the database using the Spin SDK:
pub(crate) fn insert(&self, db_url: &str) -> Result<()> {
let params = vec![
as_param(&self.id).ok_or(anyhow!("The id field is currently required for insert"))?,
ParameterValue::Str(&self.handle),
match as_param(&self.avatar) {
Some(p) => p,
None => ParameterValue::DbNull,
}
];
mysql::execute(db_url, "INSERT INTO profiles (id, handle, avatar) VALUES (?, ?, ?)", ¶ms)?;
Ok(())
}
pub(crate) fn get_by_handle(handle: &str, db_url: &str) -> Result<Profile> {
let params = vec![ParameterValue::Str(handle)];
let row_set = mysql::query(db_url, "SELECT id, handle, avatar from profiles WHERE handle = ?", ¶ms)?;
match row_set.rows.first() {
Some(row) => Profile::from_row(row),
None => Err(anyhow!("Profile not found for handle '{:?}'", handle))
}
}
pub(crate) fn update(&self, db_url: &str) -> Result<()> {
match &self.id {
Some(id) => {
let params = vec![
ParameterValue::Str(&self.handle),
as_nullable_param(&self.avatar),
ParameterValue::Str(id.as_str()),
];
mysql::execute(db_url, "UPDATE profiles SET handle=?, avatar=? WHERE id=?", ¶ms)?
},
None => {
let params = vec![
as_nullable_param(&self.avatar),
ParameterValue::Str(self.handle.as_str())
];
mysql::execute(db_url, "UPDATE profiles SET avatar=? WHERE handle=?", ¶ms)?
}
}
Ok(())
}
pub(crate) fn delete(&self, db_url: &str) -> Result<()> {
match &self.id {
Some(id) => {
let params = vec![
ParameterValue::Str(id.as_str())
];
mysql::execute(db_url, "DELETE FROM profiles WHERE id=?", ¶ms)?
},
None => {
let params = vec![
ParameterValue::Str(self.handle.as_str())
];
mysql::execute(db_url, "DELETE FROM profiles WHERE handle=?", ¶ms)?
}
}
Ok(())
}
The implementation is pretty simple, build your list of parameters and call into the Spin SDK to execute/query the database. All of the errors from the MySQL driver were pleasantly passed back to my code which made debugging relatively easy.
Those two files mentioned above contain the bulk of the “real business” logic. Let’s take a pause so you can make sure to review the code. If you have questions or comments, you can also contribute to the pull request on GitHub.
But there were a few things that I thought could be handled more ergonomically. When mapping the results back into my Profile struct I had to rely on the index of the column which in turn means that the order of columns listed in the select statement is tightly coupled to my parsing logic.
// helper mapping function
fn from_row(row: &spin_sdk::mysql::Row) -> Result<Self> {
let id = String::decode(&row[0]).ok();
let handle = String::decode(&row[1])?;
let avatar = String::decode(&row[2]).ok();
Ok(Profile {
id,
handle,
avatar,
})
}
I don’t expect Spin to map structs for me, it’s not an ORM, but if I could address the column’s by name instead of by index I could decouple the parsing logic from the query itself. As we add columns to the table in the future it could be quite tedious to maintain. Digging around the Spin SDK some more, it does seem like you can address the columns from the RowSet type. With a few tweaks I was able to get something working.
pub(crate) fn get_by_handle(handle: &str, db_url: &str) -> Result<Profile> {
let params = vec![ParameterValue::Str(handle)];
let row_set = mysql::query(db_url, "SELECT id, handle, avatar from profiles WHERE handle = ?", ¶ms)?;
let column_map = row_set
.columns
.iter()
.enumerate()
.map(|(i, c)| (c.name.as_str(), i))
.collect::<HashMap<&str, usize>>();
match row_set.rows.first() {
Some(row) => Profile::from_row(row, column_map),
None => Err(anyhow!("Profile not found for handle '{:?}'", handle))
}
}
...
fn from_row(row: &spin_sdk::mysql::Row, columns: HashMap<&str, usize>) -> Result<Self> {
let id = String::decode(&row[columns["id"]]).ok();
let handle = String::decode(&row[columns["handle"]])?;
let avatar = String::decode(&row[columns["avatar"]]).ok();
Ok(Profile {
id,
handle,
avatar,
})
}
This works, but to me, it feels like it something similar could be part of our SDK. Maybe even avoiding the need to manually create the column_map ourselves and avoid passing it to another function. I’ve filed an issue for this in Spin repository to get feedback from the engineering team on this one.
Validation
Finally our code is implemented and compiling. Let’s give it a test drive. I wrote a quick bash script scripts/validate.sh to help me perform a simple end-to-end test using curl.
#!/usr/bin/env bash
scheme="http"
host="127.0.0.1:3000"
echo "Creating the profile"
curl -v -X POST $scheme://$host/api/profile \
-H 'Content-Type: application/json' \
-d @scripts/create-profile.json
echo "------------------------------------------------------------"; echo
echo "Fetching the profile"
curl -v -X GET $scheme://$host/api/profile/justin
echo "------------------------------------------------------------"; echo
echo "Updating the avatar"
curl -v -X PUT $scheme://$host/api/profile/justin \
-H 'Content-Type: application/json' \
-d @scripts/update-profile.json
echo "------------------------------------------------------------"; echo
echo "Deleting profile"
curl -v -X DELETE $scheme://$host/api/profile/justin
echo "------------------------------------------------------------"; echo
echo "Fetching after delete should be 404"
curl -v -X GET $scheme://$host/api/profile/justin
echo "------------------------------------------------------------"; echo
Aside from me fudging some SQL queries, everything shockingly just worked! Only shocking because this was my first real attempt at Rust. Once I understand the borrowing semantics and lifecycles a little better, Rust could be a new favorite for me. If you want to play around with the code yourself you can clone the repo and checkout this feature branch:
git clone https://github.com/fermyon/code-things.git
cd code-things
git checkout feature/user-profile-api
Summary
For each of the blog posts in this series, all of the code changes I make will be contained in a single pull request so they’re easier to follow. The PR for this post is live as of January 19th, 2023 and will remain open for a week or two in case you want to review my code (see: #PR/1). I tried to be as critical as I could about the Spin SDK and I think I’ve got some good feedback for our team already.
Next time we’re going to tackle setting up the Vue.js app and integrating an authorization server to protect the API endpoints, should be fun! And did I mention this is just the beginning? 😄 Stay tuned for more blog posts as we continue our journey, check out the GitHub project to see the code for yourselves or reach out on Discord. We’d love to hear your feedback, feature requests or just chat about WebAssembly.
Read Next
Building a social app with Spin (2 of 4): Vue.js app and Token Verification
Links
- GitHub Project Links
- Thorsten Hans: CRUD in WebAssembly with Fermyon Spin and MySQL