Introducing the Spin Python SDK
Joel Dice
python
spin
sdk
Back in December, we announced the Spin JavaScript/TypeScript SDK. Today, we’re pleased to announce an experimental SDK for another wildly popular programming language: Python.
In this post, we’ll look at how to build a serverless app using Python and deploy it to Fermyon Cloud in a few simple steps. Then, we’ll dig a bit deeper and use Pipenv to add a third party dependency, as well as take advantage of the SDK’s support for outbound HTTP requests and persistent key-value storage. Finally, we’ll peek behind the scenes at how the SDK translates Python code to WebAssembly, based primarily on the incredible work the CPython team has done to support WASI.
Introduction
The Spin Python SDK consists of two parts:
- A Spin plugin, called
py2wasm, which takes a Python module and all its dependencies and produces a Spin-compatible WebAssembly module as output. You can install the plugin using spin plugin install py2wasm.
- A Spin template for quickly starting a new Python-based Spin project. This can be installed using
spin template install --git https://github.com/fermyon/spin-python-sdk.
Hello, World!
Let’s build a simple “hello, world” app starting with the http-py template.
First, install spin, the py2wasm plugin, and the http-py template:
curl -fsSL https://developer.fermyon.com/downloads/install.sh | bash
sudo mv ./spin /usr/local/bin/spin
spin plugin install py2wasm
spin templates install --git https://github.com/fermyon/spin-python-sdk
Next, use the http-py template to create a new project called hello:
spin new --accept-defaults http-py hello
Finally, build and run it:
spin build
spin up
While it’s running, use your browser or cURL in another shell to try it out:
curl -i 127.0.0.1:3000
You can edit app.py using your favorite editor or IDE, split your app into multiple modules, add new dependencies with pipenv install, etc. When you’re happy with it, you can publish your application to a registry such as Docker Hub or the GitHub Container Registry using spin registry push and run it on your own infrastructure with spin up, or else deploy it to your free Fermyon Cloud account. The following command will prompt you to log in with your GitHub account if you haven’t already, then deploy your app to Fermyon Cloud, and finally print the URL you can use to access it and share with others:
spin deploy
Beyond “Hello”
Let’s do a bit more with our app. We’ll extend it to:
- Make an outbound HTTP request to retrieve a TOML file
- Parse the TOML file using the
toml library
- Extract an item from the file and sync it with a key-value store
- Return a response body indicating whether the item changed
First, let’s add the toml package as a dependency using pipenv:
pipenv install toml
Next, we’ll edit spin.toml to give our component access to the default key-value store and specify which URL to get the TOML file from:
spin_version = "1"
authors = ["Dev Eloper <dev.eloper@example.com>"]
description = "hello"
name = "hello"
trigger = { type = "http", base = "/" }
version = "0.1.0"
[[component]]
id = "python-sdk-example"
source = "app.wasm"
# Set an environment variable to tell the code where to fetch the TOML file from:
environment = { URL = "https://raw.githubusercontent.com/fermyon/spin/main/Cargo.toml" }
# For security, components don't have permission to make outbound HTTP requests by default.
# Here we grant permission to access the host specified in the URL above:
allowed_http_hosts = ["https://raw.githubusercontent.com"]
# Again for security, components don't have access to any key-value stores by default.
# Here we grant permission to the default store for this application:
key_value_stores = ["default"]
[component.trigger]
route = "/..."
[component.build]
command = "spin py2wasm app -o app.wasm"
Now we’re ready to edit the code in app.py:
from spin_http import Request, Response, http_send
from spin_key_value import kv_open_default
from os import environ
import toml
def handle_request(request):
# First, request the TOML file from the URL specified in the "URL"
# environment variable.
response = http_send(Request("GET", environ["URL"], [], None))
# Next, parse it into a dictionary using the `toml` library.
file = toml.loads(str(response.body, "utf-8"))
# The item we want is at `workspace/package/version`.
new_version = file["workspace"]["package"]["version"]
# Now open the default key-value store and get the previous value,
# if present.
store = kv_open_default()
old_version = store.get("version")
# `store.get` returned a byte array if the value was present; let's
# convert that to a string.
if old_version:
old_version = str(old_version, "utf-8")
# If the new value differs from the old value (or we're running this
# for the first time), say so in the response and save the new value
# in the store. Otherwise, respond that there has been no change.
if new_version != old_version:
body = f"version changed from {old_version} to {new_version}\n"
store.set("version", bytes(new_version, "utf-8"))
else:
body = f"no change; version is still {old_version}\n"
return Response(200, [("content-type", "text/plain")], bytes(body, "utf-8"))
Finally, we can rebuild and rerun:
spin build
spin up
And then in another terminal:
$ curl 127.0.0.1:3000
version changed from None to 0.9.0
$ curl 127.0.0.1:3000
no change; version is still 0.9.0
To review: we’ve updated our app to check for changes in a TOML file, using Spin’s outbound HTTP API to retrieve the file, plus its built-in key-value store to keep track of the previous value.
py2wasm: Under the Hood
You might be wondering how py2wasm converts Python modules to WebAssembly. Per usual in the open source world, it relies on the hard work of the developers of several third-party projects, most notably:
- CPython: This is the standard Python interpreter, which now has first-class WASI support thanks to Christian Heimes and others.
py2wasm pairs your application module with a copy of CPython, including a trimmed-down copy of its standard library.
- PyO3: This provides bindings for interacting with CPython from Rust code, including defining native modules, types which can be shared by Rust and Python, and many other ergonomic enhancements on top of the CPython C API.
py2wasm uses these bindings to reuse the Spin Rust SDK as an idiomatic Python module.
- Wizer: This utility allows
py2wasm to pre-initialize the application module at build time, including locating dependencies, initializing global state, etc. The resulting Wasm module is primed and ready to run with negligible (sub-millisecond) startup time.
Future Work
As of this writing, the Spin Python SDK is still experimental and under development. One big missing piece is support for popular Python libraries which rely on native code, such as Pandas and NumPy. Supporting these libraries is challenging because they rely on dynamically linking native code into the CPython interpreter, which isn’t supported in core Wasm or WASI. There are ways to address this, as shown by projects like Pyodide, so we’re confident we’ll be able to do the same for Spin, but that work has not yet been done.
Another to-do item on our list is adding PEP 561 type stubs for the Spin API, which will allow IDEs and tools like MyPy to statically verify your app’s use of the API, avoiding unnecessary surprises at runtime.
In the longer term, the Bytecode Alliance is working to improve support for all dynamic languages in Wasm and the WASI Component Model, including initiatives to allow more intensive deploy-time optimizations via ahead-of-time (AOT) compilation and pre-initialization. We plan to contribute to those efforts and fold the results into Spin as they become available.
Conclusion
We’re excited to add Python to the list of languages you can use to build Spin applications, thanks to the incredible work of the CPython, PyO3, and Wizer developers, among others. Python is currently the most popular programming language in the world, widely regarded as easy to learn, and host to many popular libraries and toolkits, especially in the machine learning and data science fields. Now it’s also an easy way to get up and running with Spin and Fermyon Cloud!
Next Steps
For more details on how to get started with Spin, Fermyon Cloud, and the Python SDK, check out the following resources: