Automating API endpoint testing with Spin
Andrew Steurer
spin
sqlite
github actions
ci
ci/cd
hurl
test
testing
api
endpoint
hurl
Did you know the Spin CLI has a built-in SQLite database? Not only is it convenient to test SQLite databases and database clients locally, it simplifies running automated API tests.
What is Spin?
Spin is an open-source framework for building WebAssembly applications, and has a ton of amazing features - including portability (a single binary can run on many architectures), and sub-millisecond (0.51ms) response times, which means an application can serve requests without any noticeable delay or cold start. See our docs for more details.
Why would someone want to automate API tests?
One reason you might consider automating API tests is increase the quality of the code. Automating API tests ensures consistency in the testing process, as automated tests are less prone to human error than manual testing. This consistency can lead to more reliable and accurate results, which is crucial in maintaining the quality of your API over time. Automated tests can also be executed more frequently, enabling you to catch issues earlier in the development cycle, which can save time and resources in the long run. Finally, with automation, you can easily integrate your tests into a continuous integration/continuous deployment (CI/CD) pipeline, ensuring that your APIs are always validated before they go live.
Another reason you might consider automating API tests is to document and reduce the toil of repetitive tasks. By nature of coding an automated solution, you are documenting a step-by-step procedure, which can be very helpful when someone new is reviewing the process, or when someone has to review the code after a while away from the process. Moreover, any time you can reduce toil (work that people don’t like to do), it improves developer experience by freeing up time and mental resources to focus other tasks.
How to run API tests on a local Spin app
We have some example code that you can clone: https://github.com/fermyon/api-testing-demo
Requirements
- Latest version of Spin
- Latest version of TinyGo
- Latest version of Go supported by TinyGo
- This can be checked by running
tinygo version. At the time of this post’s publishing, the latest supported version of Go is 1.22.
- Latest version of Hurl
Building and running the application
Let’s start by building the Spin app. In your terminal, navigate to the root directory of the api-testing-demo and run the build command:
spin build
Once the build has finished, we can run the Spin application with a SQLite database:
spin up --sqlite @schema.sql
Once the application is running, open a new terminal window and navigate to the root directory of the api-testing-demo, and try interacting with the API:
-
Create some new users:
curl --request POST localhost:3000/user/user1
curl --request POST localhost:3000/user/user2
curl --request POST localhost:3000/user/user3
-
List all created users:
curl localhost:3000/all_users
-
Get a user by their ID:
curl localhost:3000/user/1
-
Delete a user:
curl --request DELETE localhost:3000/user/1
Keep in mind that the SQLite data is stored in the .spin directory. Because we have inserted data into the database, this is going to prevent our tests from succeeding, so be sure to delete the database once finished trying out the API:
rm .spin/sqlite_db.db
Testing the application
We use Hurl to automatically test the API endpoints in this application, and these can be found in the test.hurl file in the root directory of the code.
To break down the syntax of a Hurl test, let’s examine this block of code:
GET http://localhost:3000/user/1
HTTP 200
[Asserts]
jsonpath "$.id" == 1
jsonpath "$.userName" == "user1"
jsonpath "$.email" == "user1@example.com"
You’ll notice we have our HTTP method GET, and the URL of the endpoint we are trying to reach. Below this is HTTP 200, which validates whether the application returned an HTTP 200 response code. Once the HTTP status code has been validated, we then proceed to check the body of the response to see if it matches what was expected. Each line of code below [Asserts] is validating a JSON field. In this case, we are expecting a JSON response that looks like this:
{"userName": "user1", "id": 1, "email": "user1@example.com"}
Now that we understand how a Hurl test works, let’s run all of the Hurl tests that we have in test.hurl. Let’s run the Spin application:
spin up --sqlite @schema.sql
While this is running, open a new terminal window and navigate back to the root directory of the code. To run the tests, you can run this command:
hurl --test test.hurl
If you get an error that looks like this, you forgot to delete the database, so run the command rm .spin/sqlite_db.db to fix it:
error: Assert status code
--> test.hurl:6:6
|
| POST http://localhost:3000/user/user1
6 | HTTP 202
| ^^^ actual value is <400>
|
Otherwise, you should see this output:
test.hurl: Running [1/1]
test.hurl: Success (12 request(s) in 20 ms)
--------------------------------------------------------------------------------
Executed files: 1
Succeeded files: 1 (100.0%)
Failed files: 0 (0.0%)
Duration: 20 ms
This means that the test was successful.
How to run API tests on a Spin app in Github actions
Running automated tests in Github actions is very similar to running tests locally. If you examine the Makefile and .github/workflows/main.yaml files, you’ll see that we build, run, and test the Spin app. In fact, if you navigate to the root directory of the code in your terminal and run make build, make run, and make test, the tests will work the same.
We don’t have the ability to open a new terminal window in Github actions, so we have to run the spin up command in the background and redirecting the stderr and stdout to /dev/null:
spin up --sqlite @schema.sql > /dev/null 2>&1 &
Once we finish running the tests, we need to stop the Spin app from running, so we’ll kill the process running in the background. The &>/dev/null is supressing any errors we might get from the pkill command, which makes it easy to use the make rm command locally:
pkill -f 'spin up' &>/dev/null
Looking at the .github/workflows/main.yaml file, we can see that Github actions runs the make commands to test the application (after the proper software is installed):
- name: Build Spin app
run: make build
- name: Run Spin app
run: make run
# Waiting for the Spin app to be ready to receive HTTP requests
- name: wait for 1s
run: sleep 1
- name: Run Hurl tests
run: make test
- name: Stop Spin app
run: make rm
If you are looking to automate the testing of your Spin APIs, using Hurl and Github actions is a great way to do this. If you are curious about other integrations Spin has to offer, see our API guide. If you have questions or comments, we would love to hear from you on our Discord.