It’s Time to Reboot Software Development
Matt Butcher
spin
webassembly
wasm
component model
Sometimes we do things in a particular way, and become so accustomed to it that we forget to ask if there’s a better alternative. And then sometimes a change comes along that opens our eyes to an alternative. We have been writing code in a way that is not only extraordinarily wasteful but also binds us to subpar security. However, the WebAssembly (Wasm) Component Model, a new specification, presents the opportunity for a polyglot future that is more manageable, configurable and secure.
Let’s begin by tackling the significant issue of code duplication.
A 2,400-Hour Problem
Parsing URLs is a relatively benign problem in software development. The URL standard was originally introduced in 1994 as RFC 1738. While the specification has undergone a few revisions (notably the definition of the URI in RFC 2396), it has been largely unchanged. URLs and URIs are so common that even non-technical people tend to know a URL when they see one. (I’m just going to say “URL” from now on since more people are familiar with that term. You can mentally substitute “URL and URI” if you prefer.)
Parsing URLs is neither particularly esoteric nor theoretically difficult. It involves breaking down a string into constituent parts, each representing something about the location of a resource. Writing a URL parser doesn’t require novel algorithms, external resources (like networks), specialized hardware, or even libraries outside of most programming languages’ standards. In other words, while writing a URL parser is not exactly trivial, it is straightforward and non-novel.
RedMonk lists the top programming languages. Let’s take a quick sampling. JavaScript has at least two URL libraries (Node and browser, though technically, the browser library is different for each browser provider), and TypeScript also uses those. Python has urllib. Java, PHP, C#, Ruby, Dart, and Go all have one built-in URL library. PowerShell can use the .NET implementation as well. C++ has several, including POCO and Boost. C has numerous libraries. Swift and Objective-C have URL parsing provided by Apple’s libraries. R has a few (notably httr). Scala and Kotlin can use the Java version, but each also has its own. I’ve used a couple of different ones in Rust (there are several). And in UNIX shells, using regular expressions seems to be de rigueur. CSS is the lone “language” in RedMonk’s top 20 that cannot parse a URL.
All told, that’s over 30 different URL parser libraries. 30 different implementations of a basic algorithm of a rudimentary specification.
Let’s make the charitable assumption that, on average, it takes about 80 person-hours to produce a full-featured URL parser. I think this is a very, very low estimate. A quick look at, say, the Rust URL parser shows there are 80 versions comprised of 1,330 commits. I would guess the maintainers have spent far more than 80 hours. And for languages like Java, the URL library has existed for nearly three decades now. I doubt the code has seen fewer than several hundred hours of related dev time. But let’s just go with 80.
Now, we can do some back-of-the-napkin math. Producing the above 30 URL parsers at 80 hours per library means collectively, we have spent 2,400 programming hours just creating URL parsers for popular languages.
In the grand scope of software, URL parsers are easy. Complexity goes up from there. I once wrote the HTML 5 PHP parser. It took three of us well in excess of 1,000 hours. And we had the benefit of imitating the work done by others, who probably spent far more time. Yet again, I am sure we could easily list 30 different HTML parsers, all of which are dedicated to parsing the exact same format that is well-defined by specifications. If we were to delve into cryptography or networking—again, two areas where specifications are implemented repeatedly—I suspect we’d see the base time investment per library stand equally as high.
Pause to think about this for a moment: We spent vast amounts of time re-implementing the exact same specifications dozens of times.
Moreover, each time each specification is reimplemented, bugs and subtle incompatibilities are introduced. So, in addition to investing huge swaths of time, we introduce issues that have an impact on downstream usage, which in turn results in other people having to spend their time resolving inconsistencies between implementations.
Taken as a whole, this is madness. We are wasting tremendous amounts of our most valuable resource: Time.
But there is a ready objection at hand: “Wait, the reason we do this is because we have to have multiple implementations because each language needs its own implementation.”
And therein lies the problem that has now been solved.
The WebAssembly (Wasm) Component Model is a Big Deal
We have talked about many advantages to Wasm: speed, next-gen serverless, and portability. But now, the killer feature is here and ready to use. So, let’s talk about the Wasm Component Model, not in terms of how it works, but what it provides.
Wasm gives us a platform-neutral binary format that is securely sandboxed. And over two dozen languages can be compiled into it. But what if you could make it possible for these binaries to talk to each other (regardless of the original programming language)?
That’s the idea behind the Wasm Component Model: Provide a way for a given Wasm binary to express what it needs to import and also express what features it exports. And then, provide a way to say: Component A needs to import Feature X, and Component B provides Feature X, so let’s link them. For example, we can write a component that imports a URL parser, and we can write another module that exports a URL parser. The runtime then manages the mapping of the import to the export.
Here’s a video that illustrates the idea:
A model like this opens a horizon of new possibilities. Let’s explore three:
- Polyglot programming as it’s never been done
- Abstracts and interfaces as configuration
- The most secure code you’ve ever seen
Polyglot Programming as it’s Never Been Done Before
A huge and immediate benefit of the Wasm Component Model is that, since every component is just Wasm, it does not matter what the origin language of a component is.
A component written in Python can import a component written in JavaScript. A component written in Go can import a component written in Rust, which in turn can import a component written in TypeScript! Best of all, the “downstream” component doesn’t even need to know what language an “upstream” component was written in. The Wasm Component Model makes the upstream component appear as if it were written in the same language as the downstream. In other words, when I import a Rust component from a TypeScript one, it looks just like the import of a regular TypeScript library.
Think of the thousands upon thousands of hours that can be saved by not having to duplicate the same functionality, the same specs, and the same features in every single language! Think of the forward migration path when legacy code can be brought along as-is until it can be replaced. Think of the possibilities of writing performance-sensitive code in a low-level language like C, Rust, or Go while continuing to use a high-level language like JavaScript for day-to-day coding.
The Wasm Component Model is that big of a shift in the way we write code. I believe that enabling true polyglot programming is reason enough to pivot to Wasm as a compile target. But it’s not the only reason.
Abstractions and Interfaces as Configuration
Abstraction gives us flexibility. Programming languages have tools like interfaces, traits, and abstract classes so that we, as programmers, can declare the shape of an API (“I need a thing that looks like this”) without declaring the implementation (“I need exactly this thing”). For example, a SQL database interface may define a set of functions (open(), close(), exec(), query()) that a number of database drivers implement. For testing, we can mock code that imitates how we expect something to behave in production, but to do so in a way that allows us to systematically test the behaviour of our application. Even low-level system libraries like sockets, files, and streams can have different implementations depending on their environment.
Abstraction is tremendously useful when working within a single language.
The Wasm Component Model provides abstraction at the component level. That is, one component may need to import a key-value store. Another component may implement (export) a key-value store based on Redis, while yet another may implement a key-value store based on Etcd. If both the Etcd and Redis versions implement the same interface, then the consumer component can use either one. Thus, a detail of the code becomes a configuration option that can be managed outside of the code itself. (Java introduced this pattern somewhat with the class loader. You can imagine this as a more advanced classloader that doesn’t require the developer to actively manage classloading in their code.)
This leads us to the third game-changing advantage of the component model: We can write code with astounding security properties.
The Most Secure Code You’ve Ever Seen!
It’s easy to gloss over one of the details of the Wasm Component Model: How the components are executed. But when we couple this with abstraction-as-configuration, an enticing possibility takes shape.
Each component is compiled into a Wasm binary, and that binary is then linked with other Wasm binaries when an application is run. But “link” doesn’t mean “merged into.” It means that each component runs in its own sandbox, and the host environment moves data between components according to the contract established between them (via imports and exports).
Each component runs in its own sandbox.
Calling one component from another is always mediated through the host.
Combine those, and you get a startling security profile: You can layer security on every single component individually.
That YAML parser component you grabbed off the Internet? You can make sure it is not allowed to access the network. Meanwhile, you can use that authentication component to access the username and password of a user, but prevent any other component anywhere in your app from ever having access to the password. Your database driver may only be limited to local IP addresses while only the caching module can access the key-value storage. And all of this can be asserted at the configuration level! Nobody even needs to audit the code to verify this!
All of this comes with strong software supply chain guarantees as well, since components can be independently verified as well as verified together. That is, we can verify that each component has not been tampered with, as well as verify that the app as a whole has not been altered.
Combined with interfaces and abstractions, we can envision hot patching one component without the rest of the application ever being touched, firewalling off a component with a newly discovered vulnerability until a patch can be provided, or instrumenting a particular component to find out why it seems to make network calls more frequently than it should. And these features just fall out of the component model. They don’t require you to do extra work writing extra code to enable this level of security.
The sandboxed component model transforms how we can reason about the security of our applications.
Will this be easy?
Switching things over won’t be easy, but it may be quicker than we think.
First, lots of languages need to be compiled to Wasm, and this requires each language ecosystem to do some work. That is well underway. Most of the top-tier languages already have some degree of Wasm support.
Second, component tooling needs to get good. We at Fermyon are dedicated to making that happen, as are many of the contributors to the Bytecode Alliance. But for those of us in the avant-garde, we’ll have to deal with the hiccups of an early ecosystem.
Third, we collectively need to do the work of building components. This part may not be as hard as it seems, since, in many cases, it is just a matter of compiling existing code and adding bindings.
It’s time to Reboot Software Development
The Wasm Component Model opens a new horizon for polyglot programming. It has a powerful abstraction model we can use to link components together by interface, even at runtime. And because each component runs in its own security sandbox, we can build systems far more resilient and secure.
I started by pointing out how many hours are consumed duplicating the same functionality in so many different languages. That’s a problem that few of us ever thought of trying to solve. But the component model solves it.
We have an example of how to compose a middleware component (GitHub OAuth) with a business logic component. You can follow these instructions to build the components and run the demonstration.
We’ve accepted that abstractions are generally solved at compile time (with a few exceptions). However, the interface imports and exports of the component model allow us to compose and recompose applications at a configuration level.
And until now, we’ve accepted as a foregone conclusion that to use an external library means that we must blindly trust that it would not do anything nefarious. Now we can structure our applications to make them resistant to misbehaving or malicious libraries—and we can reason about these systems without needing to read the code.
Add these up, and adopting these patterns will immediately improve software development. Perhaps it’s time to reboot software development towards the component model.
Spin and Fermyon Cloud
As we have mentioned, all of the groundwork being laid in the development of the component model will enable components to be shared among mainstream languages such as C, JavaScript, TypeScript, Python, .NET, Rust, and Go. So, how does an application developer package all of this into a useful real-world application?
If you are interested in how Fermyon is putting the component model into practice, check out the Spin framework that allows you to build and run event-driven microservices with components. Aside from being able to orchestrate each of these individual components carefully, Spin also offers:
With Spin and Fermyon Cloud, you can ship your applications to our serverless cloud environment that offers custom domain implementation (with auto-renew SSL certs), serverless AI inferencing and much more.