What is Bindle
Matt Butcher
bindle
webassembly
Bindle is a package repository system designed with WebAssembly in mind. Unlike “monolithic” package managers that focus on distributing large individual binaries, Bindle provides an intelligent system for distributing groups of related objects. As WebAssembly moves toward a component-based model, Bindle makes it possible for clients to pull only the pieces they need.
Or, to think of it in a completely different way, Bindle is the silverware drawer for WebAssembly.
The Silverware Drawer
Package managers like npm, Debian’s apt system, and even Perl’s venerable old CPAN all have something in common: They distribute packages as big compressed files. Other package managers, like Go’s, deliver software as source code clones of a Git repository, and rely on the developers to build what they want.
Bindle takes a different approach.
In Bindle, a package (called a bindle, with a lowercase b) has a top level invoice that explains what the package does and lists all of the files that are part of that package. These files are packaged as parcels, smaller objects which can be retrieved individually. A bindle is made up of exactly one invoice, but any number of parcels.
But it’s not this fragmented format that makes Bindle interesting. It’s that Bindle is designed so that the client can fetch the invoice, see which parts it needs, and then fetch only the pieces of the package that it requires.
To explain this, the Bindle documentation uses the example of the silverware drawer:
Consider the humble silverware drawer. When we set the table for dinner, it’s convenient to open one drawer and get the forks, spoons, and knives. Yes, a fork is a different thing than a knife. Yes, there are multiple different kinds of spoons. And, yes, silverware is not even uniform in size or shape brand-to-brand, model-to-model. Some people keep chopsticks with the silverware. Others toss in those tiny spreader things you use to slather on the cream cheese at a fancy party. In my house we keep the straws in the silverware drawer. Drawers are flexible. They can accommodate these variances.
Bindle is designed to accommodate all kinds of objects – WebAssembly modules, text, JavaScript, supporting images or CSS, videos… whatever. And the invoice file explains what is stored in a particular bindle. The term for this kind of mechanism is aggregate object storage. Again, from Bindle’s documentation:
More specifically, Bindle is an aggregate object storage system. Merriam-Webster defines an aggregate as “a mass or body of units or parts somewhat loosely associated with one another.” The fundamental feature of Bindle is that it provides you with a way to group your associated objects into an organized and named unit. It thinks in terms of aggregates.
But an invoice is more than just a list of opaque blobs. Parcels can be annotated with groups and features. Groups explain how two or more parcels are related. Extending the silverware drawer, the parcels for fork, spoon, and knife might be grouped together for the casual dinner group. Parcels can be members of more than one group, too. So the breakfast cereal group may also have spoon (though not a fork and knife).
In contrast to groups, features allow a bindle to mark that a parcel is only used for a specific reason. A soup spoon might be associated with a liquid in a bowl feature, meaning that it was only used for consuming liquids that are in bowls. In a more technical example, we might use features to indicate that a parcel requires a GPU or is only required on Windows.
Features and Groups in Action
Using a combination of groups and features can allow a developer to describe complex interdependencies between WebAssembly modules. For example, it can express a relationship like this:
The application can be configured to use one of three different databases (PostgreSQL, SQLite, or MySQL), though if MySql is selected, the application also needs an additional helper library. And if it is run on a system that does not support a GPU, then it needs a special shim library.
In this hypothetical example, the bindle would contain the main app, and also the three Wasm components for each of the databases, as well as the additional Wasm component for MySql. And, it contains the extra component that is to be used if the host system does not have a GPU. A package like this could get big! But not to worry. As we will see, the client only pulls the packages that it needs.
Here’s the list of things our bindle contains:
- main app
- Group: Select one of...
- database 1 (postgres)
- database 2 (sqlite)
- database 3 (mysql)
- helper for mysql
- Feature: No GPU is present
- Shim library
All of that information is stored in the bindle. And when the client application pulls the bindle, it can reason about its local configuration (“I support one of the databases, PostgreSQL. And I don’t have GPU support, so I need that extra shim package”) and then fetch only the parcels it needs to run under those circumstances.
✅ main app
- Group: Select one of...
✅ database 1 (postgres)
- database 2 (sqlite)
- database 3 (mysql)
- helper for mysql
- Feature: If GPU is not present
✅ Shim library
Since the client can calculate based on the invoice alone, it does not need to pull any of the parcels until it has determined which parcels it needs. So in the example above, only three of the total six parcels need to be fetched from the Bindle server. This can save substantial network overhead and storage.
Bindle allows developers to express all the possible configurations for running an application, and allow the operator to decide (at startup time) which specific configuration will be run.
But wait! There’s more! In a world full of open source libraries, it is likely that multiple applications will share at least some parcels. So the client can do some additional calculating.
Say a new package is installed, and it has a couple of parcels: The main app code (second app) and its database component postgres.
- second app
- postgres
The bindle client can look at the above and say, “Hey, I already installed the exact same version of postgres for main app. I don’t need to download that again.” So when it fetches parcels, it just needs one:
✅ second app
- postgres
And again, we’ve saved network bandwidth, sped things up, and saved storage space!
Admittedly, these examples are a little bit simplified (Bindle makes much stronger guarantees about the invoices and parcels). You can take a look at 10 more realistic examples in the documentation, or read the Bindle specification to learn about its security properties and structural elements.
Or if you’d like a more visual tour, you can check out this video from Wasm Day North America 2021:
Conclusion
Bindle is aggregate object storage. It’s a silverware drawer for objects, and it’s designed with WebAssembly in mind. It provides an efficient and flexible packaging system for component-based applications that have variable dependencies. Using Bindle, developers can create packages that describe all the possible configurations of an app. And when a user or operator fetches the app, they can use groups and features to choose a particular configuration they want.