Blog product

August 24, 2023

TypeScript and Fermyon Cloud Key Value Storage

Matt Butcher

Cloud NoOps Serverless Storage

Traditionally, serverless functions are stateless. But we always need a place to store some state. Fermyon Cloud and Spin support a few NoOps options. One is to use our built-in SQL database. But an even simpler storage mechanism is Key/Value storage. In this article, we will take a look at how to use each of the Key/Value storage functions in the TypeScript and JavaScript SDK, with emphasis on how to work with JSON data.

💡 Check out our talk Running TypeScript in WebAssembly on the Cloud at TypeScript Congress 2023 on Sept. 21 & 22.

If you would like to follow along, with the coding, please go ahead and install Spin.

Upgrading Spin: If you have Spin installed and are interested in checking your version and possibly upgrading, please see the Spin upgrade page of the developer documentation.

Please visit our Building Spin Components in JavaScript documentation to ensure your system can compile JavaScript programs to Spin components.

Creating a New TypeScript Project

The first step is to create a new TypeScript project using spin new, and then initialize the project:

$ spin new http-ts kv-example --accept-defaults
$ cd kv-example
$ npm install

At this point, you will have a basic TypeScript application and all of the libraries you need to write Spin applications.

Because we know that we are going to be using a Key/Value database (KV for short), we should go ahead and turn on the default database in the spin.toml file:

spin_manifest_version = "1"
authors = ["Matt Butcher <matt.butcher@fermyon.com>"]
description = ""
name = "kv-example"
trigger = { type = "http", base = "/" }
version = "0.1.0"

[[component]]
id = "kv-example"
source = "target/kv-example.wasm"
exclude_files = ["**/node_modules"]
key_value_stores = ["default"] # <-- ADD THIS LINE
[component.trigger]
route = "/..."
[component.build]
command = "npm run build"

The only thing we added there is the key_value_stores = ["default"] line. Note that it is important that this line go after the [[component]], but before the [component.trigger].

Using KV Storage

There are two things we always need to do in order to use KV:

• Import the Kv object.
• Open the database.

We declared a default database in our spin.toml function, and that is the one we will use.

Note that Fermyon Cloud currently only supports a default database. Custom KV databases will come along at a later date.

Here’s our src/index.ts file:

// Import 'Kv' along with the HTTP stuff
import {
    Kv,
    HandleRequest,
    HttpRequest,
    HttpResponse
} from "@fermyon/spin-sdk"

export const handleRequest: HandleRequest = async function(request: HttpRequest): Promise < HttpResponse > {
    // Open storage.
    let store = Kv.openDefault()
    return {
        status: 200,
        body: "Hello"
    }
}

On the first line we imported Kv, and then a few lines later we opened storage with Kv.openDefault().

Now we’re ready to look at the basic functions. If you want to, you can quickly build and run the initial application, as is:

$ spin build --up

At this early stage, you will just get a response of Hello, as shown below:

$ curl -i localhost:3000                                                              
HTTP/1.1 200 OK

Hello

Simple Set, Get, and Delete Functions

Let’s dive into the basics. Here’s an example of setting, getting, and then deleting a single key:

// Import 'Kv' along with the HTTP stuff
import {
    Kv,
    HandleRequest,
    HttpRequest,
    HttpResponse
} from "@fermyon/spin-sdk"

export const handleRequest: HandleRequest = async function(request: HttpRequest): Promise < HttpResponse > {
    // Open storage.
    let store = Kv.openDefault()

    // Set a string/string pair
    store.set("name", "Pet Database")

    // Get the result and store it as a string
    let dec = new TextDecoder("utf-8")
    let name = store.get("name")
    let out = "Site name: " + dec.decode(name)

    // Delete the KV pair
    store.delete("name")

    return {
        status: 200,
        body: out
    }
}

Let’s build and run this application so we can test its new functionality:

$ spin build --up

A new request to the application can be performed, as follows:

curl -i localhost:3000                                        
HTTP/1.1 200 OK

Site name: Pet Database            

It’s important to note that while we store a KV pair of strings (store.set("name", "Pet Database")), what we get back with store.get("name") is an ArrayBuffer. So we need to convert that back into a string. We use the built-in TextDecoder to do that.

After we fetched the data back out using get(), we deleted the key using store.delete("name").

This was a simple example of how to work with strings. However, the TypeScript SDK also supports working directly with JSON data. So let’s replace the above with an example of setting and getting JSON data.

Working with JSON Data

In this example, we’ll create a simple pet database in JSON. We’ll declare an array of objects, then store it to KV, then fetch it back out and print the name of the second pet:

// Import 'Kv' along with the HTTP stuff
import {
    Kv,
    HandleRequest,
    HttpRequest,
    HttpResponse
} from "@fermyon/spin-sdk"

export const handleRequest: HandleRequest = async function(request: HttpRequest): Promise < HttpResponse > {
    // Open storage.
    let store = Kv.openDefault()

    // Create an entry with a string key and a JSON body:
    let myPets = [{
            name: "Tiberius",
            type: "dog"
        },
        {
            name: "Julius",
            type: "cat"
        },
        {
            name: "Toez",
            type: "cat"
        }
    ]
    store.setJson("pets", myPets)

    // Now get the object back as JSON:
    let petList = store.getJson("pets")
    let out = "The second pet is named " + petList[1].name

    return {
        status: 200,
        body: out
    }
}

We store the myPets object as JSON, and then retrieve it. If we run the above and then use curl to get the results, we’ll see this:

$ curl -i localhost:3000
HTTP/1.1 200 OK

The second pet is named Julius   

Underneath the hood, what Fermyon is doing is serializing the myPets object into JSON inside of the setJson() function, and then when getJson() is called, the data is deserialized back into an object. That means two things:

• We could use get() instead of getJson(), in which case we would get an ArrayBuffer containing the serialized JSON data
• If we used getJson() on a KV pair that is not JSON, it will throw an error

That second case raises an interesting question: How do we handle error cases with KV? The best way is to use the regular old try/catch blocks. So, for example, we could do something like this:

const encoder = new TextEncoder()

// --snip --

  try {
      const petList  = store.getJson("some-key")
      return {
          status: 200,
          body: encoder.encode(petList[1].name).buffer
      }
  } catch (e: any) {
      console.log("Error parsing JSON: " + e)
      return {
          status: 500,
          body: encoder.encode("Internal error!").buffer
      }
  }

The setJson and getJson combo is a nice way to work with structured data in KV. If something goes awry using the try/catch blocks gives us some control over the response. Of course, you can tailor the response to suit your own needs i.e. decide what type of error[s] to catch and then generate the appropriate message[s] and response code[s].

A Few Useful Functions

In addition to the basic open(), set(), get(), delete() functions and their JSON helpers, there are a few other useful functions:

• getKeys() returns all of the keys in a KV, which is great for looping over all of the data.
• exists() is a quick way to test whether a particular key exists in the KV without having to fetch the value.

Here are some short examples of each.

This is how to get all of the keys and then join them into a single string and log the results:

// We can get a list of all of the keys out of the DB.
let keys = store.getKeys().join(", ")
console.log("The keys in the DB are: " + keys)

And likewise, this is how to quickly test for the existence of a key, and log a warning if a key is not found:

if (!store.exists("some-key")) {
    console.log("expected 'some-key' to exist in KV")
}

In my experience, exists() can be particularly useful if the value is large (like an image). But if you plan on working with the value, it’s often faster to do one get() and check the result rather than using both an exists() and a get() (which causes two trips to the KV store):

let value = store.get("some-key")
if (value == null) {
    console.log("some-key doesn't exist")
} else {
    // do something with value
}

KV is a nice database to work with because, as we’ve seen, there are only a handful of functions we need to know, yet we can store some fairly sophisticated things.

Learn More about TypeScript, Spin, and Fermyon Cloud

Fermyon will be presenting at this year’s TypeScript Congress (100% virtual) on Sept 21 @ 11:05 PT. You can also check out our TypeScript and JavaScript developer docs. And, of course, there are many samples and templates in the Spin Up Hub.