Why Do We Need Another Kind of Cloud Compute?
Matt Butcher
serverless
faas
containers
vms
cloud computing
Sometimes you start with a technology, and look for problems that the technology solves. In many regards, this was how I spent much of my time in the Kubernetes ecosystem. Other times, though, you start with a problem and hunt for a solution. And sometimes the solution that shows itself is surprising. That’s the story when it comes to WebAssembly and the cloud.
I like to think of WebAssembly as an enabling technology for the next wave of cloud computing. To understand why, and what problems it solves, let’s start by looking at just the compute part of the cloud.
I’d Like to Rent Your CPU
Cloud compute is different than cloud as a whole. Compute makes up only a part of the bigger cloud story.
Cloud compute describes the part of the Infrastructure-as-a-Service (IaaS) layer in which the guest (user, customer) pays the host (cloud provider) to run a workload on the guest’s behalf. The workload may be as small as a serverless function and as big as an entire operating system, but in each case it represents a discrete executable unit. We could reduce this to the simple statement that cloud compute allows a guest to rent the host’s computing power.
Other IaaS offerings, like storage and networking, are not properly part of cloud compute. But they are similar. Storage services allow guests to rent a host’s storage space. Networking as a service allows guests to rent networks. And together, these IaaS services provide the building blocks upon which we can run web applications, microservices, and a host of other kinds of applications.
IaaS services are building blocks that platform engineers and devops teams can use to build systems.
For teams that don’t want to build from scratch, there are a host of higher-level services that are ready-made. Platform-as-a-Service (PaaS) lets developers quickly deploy code without composing the IaaS layer. Hosted storage services like Database-as-a-Service (DBaaS) or cloud message queueing provide a convenience layer on top of virtual machines and storage. Properly speaking, offerings like PaaS and DBaaS are not “kinds of cloud compute.” They are ways that cloud compute infrastructure is applied to solve specific problems.
The First Two Kinds of Cloud Compute
At this point, we’ve drawn some lines around IaaS, distinguishing it from higher level offerings like PaaS. And within the IaaS layer, we’ve identified cloud compute as distinct from storage, networking, and perhaps other IaaS services. Now let’s delve into the kinds of cloud compute that are widely available today.
At this moment in time, there are two widely deployed compute services: Virtual Machines and Containers. (Adding WebAssembly makes a third.)
The first type of cloud computing is the older one. Virtual Machines (VMs) were the first compute offering at AWS and elsewhere. VMs represent the heavyweight class of cloud computing. It is battle-tested and secure, and used broadly.
Security for VMs is stellar. Governments, health care, and other sensitive industries rely on VMs. Because a VM image encompasses an entire operating system from kernel to applications, it can be used for everything from hosting web servers to providing in-cloud desktop environments. The downside of packaging an entire operating system, though, is that the image size is often very large — often gigabytes in size. As a result, they are difficult to move around and slow to start up. And when operating a virtual machine, a platform engineer must take on the responsibility of maintaining the entire operating system.
The second class of cloud compute is OCI containers. We often call them Docker Containers because the core technology was developed by Docker. While a VM encapsulates the entire OS, a container is composed of only a slice of an operating system. Containers share the host kernel, and do not package their own kernel and drivers. By convention, a container only holds one server or primary process. And a container only needs the files, utilities, and libraries that the main server uses.
Because more of the operational work is delegated to the host environment, containers are often easier to manage than VMs. However, because the kernel is shared between the host and the containers, there is a wider attack surface for containers than for VMs. Thus they are not considered as secure.
While virtual machines take minutes to start up, containers take seconds. And while the average VM image is measured in gigabytes, the average container is only a few hundred megabytes of data.
Many higher-level cloud offerings are built atop VMs and containers. DBaaS systems typically use virtual machines to host the server. Managed Kubernetes provides an environment for executing clusters of containers. Underneath most of the services in your cloud’s service catalog, virtual machines and containers are powering the compute portion.
This includes one place where neither VMs nor containers are particularly well suited: Serverless functions.
Serverless Functions and the Limits of VMs and Containers
Serverless functions provide a way to run small special-purpose programs that are designed to start, execute a specific task, and then shut back down again. Sometimes we call such systems Functions-as-a-Service (FaaS). Or sometimes we call them Lambdas, using the name AWS gave them much as we sometimes call tissues “Kleenex.”
Serverless functions are a popular, and growing, paradigm. Amazon reports that they execute over 10 trillion functions per month. And CNCF’s annual survey says that by the end of 2022, 53% of surveyed developers say they have used serverless functions, up from only 37% in 2021. They are used for everything from web backends to log file processors. Unlike containers and VMs, serverless functions platforms typically support a select set of programming languages (though some provide facilities for bringing your own language).
The main attraction of a serverless function is the execution model. Functions run on-demand, often completing in less than a second.
Applications running in virtual machines or containers tend to be servers, designed to run for hours, days, and months at a time. Slow startup time is no big deal, because we plan on keeping them around.
Serverless functions, on the other hand, are distinct because we don’t run them all the time. Instead, we run a function on demand, and typically only for short periods of time. For example, when a browser requests a website backed by a function, the platform starts the function, hands it the request, and then sends the function’s response back to the browser. Then the function is terminated. This process likely lasts no more than a second or two.
This model is cheaper in many cases than longer-running servers. The code is also substantially easier to write, since there’s no need to write an entire web server just to handle a single request. As the name implies, a serverless function may be no more complex than a single function (though complexity is up to the developer).
It may come as a surprise, then, that when it comes to today’s serverless functions platforms, the biggest drawback is performance. Many serverless functions require hundreds of milliseconds to start up. That’s a few hundred milliseconds that one must wait before the code even starts running. Anyone who has done website performance work knows that this is far too long. Moreover, while serverless functions platforms target specific programming languages, the process for developing and deploying applications is often cumbersome and frustrating. Additionally, in spite of all the talk of “serverless,” most serverless function platforms require the developer to make a lot of server-specific choices. Operating system and CPU architecture must both be accounted for: if you’re running your function on an Arm64 Linux platform, you’ll need to write your code specifically for that combination. And finally, one must pay close attention to cost. Because while functions are cheaper than hosting a low-traffic long running server, that calculus can flip when traffic gets too high or the function uses too many system resources.
The reason for these frustrations lurks beneath the surface: In today’s most frequently used platforms, serverless functions are merely workloads running on virtual machines (or, in a few rare cases, containers). Each serverless function requires its own virtual machine (or container). While the compute type is hidden from the user, it is there, and its own runtime characteristics impact all aspects of the serverless function.
Above, we saw some of the drawbacks to virtual machines and containers. Both are, on the scale of milliseconds, tremendously slow to start. Both expose the underlying OS and architecture to varying degrees. To work around these, serverless functions platforms pre-initialized resources in order to come anywhere close to the performance profile that a serverless function needs. Consequently, the cost of keeping all of those pre-warmed virtual machines and containers around adds to the usage cost. You pay more.
This is where WebAssembly shows itself as a promising cloud compute technology that could underpin serverless functions platforms.
WebAssembly as a Third Type of Cloud Compute
Serverless functions are great for many reasons. I, for one, find the programming model delightful: I can dive right into the code I care about, and not have to manage long-running servers with sophisticated thread pools and tedious boilerplate code. And it appears that I am but one among a growing trend.
Serverless functions would be even better (and more broadly useful) if we could remove some of the drawbacks.
WebAssembly happens to provide just the set of features that a serverless functions cloud compute engine would need:
- Security: The WebAssembly sandbox is secure, built to be even more strict than the JavaScript sandbox in the browser or the shared-kernel container model in the cloud.
- Performance: WebAssembly was built from the ground up to be fast: fast to transfer over a network, fast to load, and fast to cold start, and fast to execute.
- OS and Architecture Agnostic: The same WebAssembly binary can run on Windows, on Linux, and on many other operating systems. Likewise, Intel, Arm, and a variety of other processor architectures work fine. The developer does not need to know or care about the environment on which the WebAssembly binary will run.
- Language Neutral: A growing number of languages, including C, Rust, JavaScript, Go, C#, Swift, Python, PHP, and Ruby, can all run in a WebAssembly runtime.
WebAssembly’s virtues make it the perfect underpinning for serverless functions.
Point for point, these meet the needs of a fast, secure, and flexible serverless runtime. In our own experiments, we’ve regularly been able to cold start a WebAssembly executable in a millisecond or less—instantaneously from the user’s perspective. A WebAssembly module can be started, executed to completion, and shut down well before even the fastest VM-based serverless functions can even load a function. It is not only fast, but easy on system resources, using a fraction of the memory and CPU of a container or virtual machine. And because there is a single binary format for all languages, the developer experience is much better. Regardless of language, a WebAssembly-based serverless functions platform can provide a consistent and predictable developer experience.
Finally, when it comes to security, WebAssembly applications can be safely executed in a multi-tenant environment. Instead of one virtual machine per serverless function, many serverless functions can be executed in the same WebAssembly runtime. Consequently, we have run thousands of WebAssembly serverless functions on relatively spartan systems.
Conclusion
Cloud compute is a key IaaS technology. In essence, it is what allows us to rent someone else’s hardware to execute our own code. Virtual Machines were the first type of cloud computing, and they remain the robust heavyweight of cloud compute. Containers are the middleweight class. With them, we can easily execute servers and long-running applications. But each has limitations that make them an ill fit for serverless functions.
WebAssembly is the third kind of cloud compute. Its performance profile and speed make it excellent for serverless functions, and its universal binary format and integration with developer tooling lowers the productivity barrier: Developers can get more done, faster. Finally, because WebAssembly services can safely host thousands of applications without a lengthy queue of pre-warmed VMs or containers, the cost of serverless is lower.
Virtual Machines, containers, and now WebAssembly form the triumvirate of cloud compute engines that we need to support today’s variety of cloud services.