LocalStack is an open-sourced server application that lets developers emulate Amazon AWS Services on their local development environment. It can be used as part of one’s development environment, but also as a way to support the running of E2E tests, first in the local environment, then in the CI pipeline.
LocalStack in the local environment
E2E tests are usually run in the local environment first, and in this case all we need to do is simply make LocalStack available for use on localhost. There are multiple ways to do this, but for our purposes, we can make use of a dockerised LocalStack application:
docker run --rm -it -p 4566:4566 -p 4571:4571 localstack/localstack
This command will expose LocalStack on a range of ports in localhost. Running E2E tests against them is as simple as knowing which ports to set for the respective AWS endpoints in your testing application.
For example, to utilise LocalStack’s S3 Endpoint, we can use the port 4566.
The above also lets me alert the reader to the fact that there are indeed dockerised versions of LocalStack – which we can potentially use in our CI pipeline.
LocalStack in Gitlab CI
For those who don’t know, Gitlab CI is a Continuous Integration and Continuous Deployment (CI/CD) pipeline, linked directly to Gitlab’s repository product. Gitlab CI allows users to immediately build and deploy code, once code is push or merged into a repository.
Each Gitlab CI pipeline consists of a set of stages, each of which can contain one or more jobs. Stages run in a sequence, while jobs can run in parallel within each stage.
The example job
One of the key features of Gitlab CI is that each job runs in its own self-contained environment. Each job consists of spinning up a dockerised application, then running a series of commands against them.
Let’s assume that we have a NodeJS server that we need to run a set of E2E tests against. We can structure this as a job:
e2e: # this is the job
stage: test # this is the stage
image: node:14 # the docker image
script: # each bullet is a bash command
- yarn # install dependencies
- yarn e2e # run the tests
variables: # variables for the node environment
PORT: "3000"
LocalStack integration
Unfortunately, once each job terminates, all instances created within the application also terminate with it. Jobs are also unable to communicate with each other. This means that we cannot simply spin up a LocalStack image with one job, to be used by another job.
It is also bad practice to maintain an external LocalStack server to run E2E tests against, because this means that test suites might potentially not run independently of each other.
Instead, what we need to do is to spin up an instance of LocalStack within the job itself. Fortunately for us, Gitlab CI allows us to do this through something called a service. Services are Docker images that are pulled as part of a job, and run within the job itself – exactly what we need!
e2e:
stage: test
image: node:14
script:
- yarn
- yarn e2e
services:
- name: localstack/localstack # LocalStack docker image
alias: localstack # service url's origin
variables:
PORT: "3000"
AWS_S3: "localstack:4566" # access via origin:port
Using the above, we can pull the LocalStack docker image (the same one we used in our local environment) and spin it up as a server within the job. By defining an alias, we define the origin at which we can access the endpoint (within the job).
We then use this origin as part of the AWS S3 endpoint url, which we can communicate to the application via environment variables.