Wasm, Kubernetes, and Sustainable AI at Civo Navigate NA 24
Matt Butcher
cloud
civo
conference
workshop
In mid-February, Civo fans descended on the University of Texas - Austin campus for the two-day Civo Navigate conference. As with London last year, the main themes in Austin were related to AI and efficiency (including WebAssembly).
We hosted a workshop on how to build WebAssembly-based Spin applications and then deploy them into a Civo Kubernetes. Kubernetes is the orchestrator of choice for the Docker container ecosystem. But it’s also an awesome way to run WebAssembly serverless functions side-by-side with your containers.
Civo’s Kubernetes (based on containerd) works seamlessly with WebAssembly once the runwasi containerd shim is installed. In our workshop, we started by creating a local k3d cluster and running Spin apps there, then we provisioned a Civo Kubernetes instance and installed and ran the same Spin apps in that cluster.
After the workshop, we had a chance to visit the Wasm Austin Weekly Happy Hour. This low key event was a great way to unwind and talk WebAssembly. (It was also my first time trying natural wine!) We got to spend time with old friends and meet some new ones.
On the second day of the conference, we talked about AI inferencing from WebAssembly. We showed a simple application that did sentiment analysis powered by the LLaMa2 model, and then we tested the app locally (running the inference on CPUs). Even that simple inference took a few minutes. So then we deployed the same binary file to Fermyon Cloud, which uses Civo’s powerful AI-grade GPUs, and we saw the same inference render almost instantly.
The goal of Fermyon’s Serverless AI is to give developers such a great developer experience that it’s easy to write and deploy AI-powered apps. But we also want to do this in a sustainable way. So in the last part of our talk, we showcased how Fermyon Serverless AI runs using Deep Green’s GPUs at Civo, which draw heat (the waste product) off of GPUs and use that heat to warm swimming pools. This is an ingenious innovation that combats the trend to just accept and mitigate GPU heat instead of re-using it.
Kubernetes and AI are crucial pieces of the WebAssembly story. We’re looking forward to Wasm I/O in Barcelona and KubeCon + CloudNativeCon EU ‘24 in Paris this month where we will be making some new announcements. We hope to see you at one or both of these fantastic events.