Repository: govuk-load-testing
Test plans for load testing GOV.UK apps using Gatling.
- GitHub
- govuk-load-testing
- Ownership
- #govuk-platform-security-reliability-team
- Hosting
- N/A
- Category
- Utilities
README
Test plans for load testing GOV.UK frontend apps using Gatling.
Table of Contents
- Terminology
-
Methods
2.1 GOV.UK Gatling
2.2 Virtual Machine
2.3 Rental of an enterprise Gatling Instance via AWS Marketplace - Configuration Options
- Simulation Plans
- Uploading
- Combining concurrent test results
- Troubleshooting
1. Terminology
-
$GATLING_HOME
is the directory where Gatling is installed to. For example, if you download version 3.0.0-RC4 of the Gatling bundle zip and extract it in your~/Downloads
folder,$GATLING_HOME
is~/Downloads/gatling-charts-highcharts-bundle-3.0.0-RC4
-
simulation plan: a set of scenarios, where each scenario represents how a user will interact with the website.
2. Methods
There are 3 main methods to install and run Gatling:
- terraform a gatling instance using GOV.UK tools and run Gatling
- install and run Gatling on a virtual machine which can reside on your laptop or a AWS instance
- rent an enterprise Gatling instance via AWS Marketplace
2.1 GOV.UK Gatling
Provision GOV.UK Gatling
To provision a Gatling instance, deploy
Terraform
for app-gatling
in the blue
stack in the required environment.
Running a Plan
-
SSH into the Gatling instance
-
Switch to the
gatling
user and into the right directorysudo su - gatling cd ~/govuk-load-testing
-
Export the required environment variables for your load test
-
Run Gatling with
/usr/local/bin/gatling/bin/gatling.sh -sf src/test/scala
-
Learn what each simualtion plan tests and choose one:
Choose a simulation number: [0] govuk.Frontend [1] govuk.WhitehallPublishing [2] govuk.WhitehallPublishingCollections
-
Now, the Gatling web UI is available at
https://gatling.<environment>.govuk.digital
. There’s an index of all Gatling runs made since provisioning, and you can see the results of a particular run.
When You’ve Finished
Manually “stop” the Gatling instance in the AWS Console UI.
2.2 Virtual Machine
Installation Steps
The installation steps are:
- Install a JDK, Gatling needs at least version 8
- Clone this repository into your
~/govuk/
directory - Download and extract Gatling, these test plans are written for version 3
- Rename and move the extracted Gatling directory to
~/govuk/gatling
to make it available in/var/govuk/gatling
in your Virtual Machine - Set the needed environment variables:
-
$ export 'JAVA_OPTS=<required-options>'
(see Configuration Options) -
$ export 'GATLING_USERNAME=<test-user-email>'
-
$ export 'GATLING_PASSWORD=<test-user-password>'
Note:
GATLING_USERNAME
andGATLING_PASSWORD
are only needed for test plans using Signon. These are the credentials for the test user that has been set up in Signon in the environment it’s going to test in, e.g.staging
.
Running a Plan
In order to run a simulation plan, Gatling provides a wrapper script to compile and launch test plans in its user-files
directory.
You can run Gatling by:
- running the following command:
$ $GATLING_HOME/bin/gatling.sh -sf src/test/scala
Note: This command must be run from within ~/govuk/govuk-load-testing
(or /var/govuk/govuk-load-testing
if you are using the Virtual Machine).
- selecting the simulation plan number that you wish to run.
Choose a simulation number:
[0] govuk.BusinessReadinessFinder
[1] govuk.DynamicLists
[2] govuk.DynamicListsEmailSignup
[3] govuk.Frontend
[4] govuk.PublishToPublishingApi
[5] govuk.WhitehallPublishing
[6] govuk.WhitehallPublishingCollections
A description of relevant simulation plans available for the gov.uk website is available here
2.3 Rental of an enterprise Gatling Instance via AWS Marketplace
Note: Gatling FrontLine instances cost $9/hour, so it’s important to switch off the instance while it’s not in use.
Launching a Gatling instance
To launch a Gatling instance, follow the instructions below. You may find there is already an existing Gatling instance available in EC2, in which case you can just start it without having to create a new one.
- First make sure you are logged in to AWS and have switched your role to production
- Go to “AWS Marketplace Solutions” from the services list (you may need to switch region to N. Virginia).
- Select “Manage” on Gatling FrontLine, click on “Actions” and then “Launch new instance.”
- Set the region to “eu-west-1” and click “Continue to Launch”.
- Select your desired EC2 Instance Type (recommended
t2.2xlarge
). - Choose the
vpc-07069e8dd026cc725
VPC,subnet-00103e6927dd1fb36
subnet andgovuk_gatling_access
security group. - Click “Launch” and you will be given a link to the EC2 instance.
- Find the Public DNS name for that instance and go to it in your browser. It should provide you with a wizard to complete the set up of the instance.
- You may find it useful to rename the instance in AWS to
gatling
so you can find it again easily. - If running the test cross-environment (e.g. running a load test in staging from a production EC2 instance), edit the
govuk_cache_external_elb_access
security group in the environment you are testing to permit access on port 443 from the public IP address of your Gatling EC2 instance.
Running a Plan
Once you have a Gatling FrontLine EC2 instance, you can use it to load a plan.
- Click on “Create” at the top left to create your plan.
- Give it a name, and choose the classname that corresponds to the test plan class you want. For example,
govuk.Frontend
for Frontend.scala. See this section for a list of plans. - Click next and choose “Build from sources” (should be the default option). Enter
git clone https://github.com/alphagov/govuk-load-testing.git
into the repository command box, chooseSBT Project
in the build command drop down and click next. - If you’re only using this one instance, choose the “Local” pool with a weight of 100%.
- Click on “More options” and here you can enter the
JAVA_OPTS
value in the second box. For example, to use 100 workers, you would enter-Dworkers=100
. Please see section for further configuration options. - Now you can click save and your plan should appear in the list. Click the play button to build it.
- Once your plan has built, it will go ahead and run it for you. You can click on the icon of a graph to view live updating results from the load test.
3. Configuration Options
We use Java properties to pass options to the script which we don’t want to hard-code. These can be set using the JAVA_OPTS
environment variable:
$ export JAVA_OPTS="-Dkey1=value1 -Dkey2=value2 ..."
The following property is required:
-
baseUrl
, prepended to all requests, at the least it should include the scheme and domain name
The following properties are necessary depending on the environment or scenario:
-
username
, the HTTP basic auth username -
password
, the HTTP basic auth password
The property signonUrl
and environment variables GATLING_USERNAME
and GATLING_PASSWORD
are required for scenarios authenticating with a signon application.
The following properties are optional:
-
dataDir
(default: “src/test/resources/test-data”), the directory to look in for test data files -
rateLimitToken
(default: no header sent), the value of theRate-Limit-Token
header -
workers
(default: 1), the number of threads making requests -
ramp
(default: 0), the duration, in seconds, over which the workers are started -
bust
(default: false), whether to pass a unique cache-busting string with every request or not -
maxTime
(default: 3600), the longest a test can run, defaulted to 1 hour
4. Simulation Plans
The simulation plans for gov.uk are located in the src/test/scala
directory of this repository
while their data files live in the src/test/resources
directory.
-
govuk.BusinessReadinessFinder
Data files: business-readiness-paths.csv
Optional:
factor
(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits
, each worker requestsbase_path
ceil(hits * factor / workers)
times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration
(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time. -
govuk.DynamicLists
Data files: get-ready-brexit-check_paths.csv
Optional:
factor
(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits
, each worker requestsbase_path
ceil(hits * factor / workers)
times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration
(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time. -
govuk.DynamicListsEmailSignup
Data files: get-ready-brexit-check-email-signup_paths.csv
Optional:
factor
(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits
, each worker requestsbase_path
ceil(hits * factor / workers)
times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration
(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time. -
govuk.Frontend
Data files: paths.csv
Optional:
factor
(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits
, each worker requestsbase_path
ceil(hits * factor / workers)
times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration
(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time.If you are having difficulty running the entire test plan on a single machine within your desired duration, try splitting up the data file and running multiple instances of Gatling simultaneously on different machines.
-
govuk.PublishToPublishingApi
Requires:
BEARER_TOKEN
environment variables.Note: The Publishing API is not accessible from the outside, you’ll need to set up a proxy into our infrastructure to connect to it from Gatling.
For example, for staging:
$ ssh publishing-api-1.staging -CNL 8443:publishing-api.staging.publishing.service.gov.uk:443
You’ll then need to add the line
127.0.0.1 publishing-api.staging.publishing.service.gov.uk
to/etc/hosts
to make sure the HTTPS server name matches.Then you can set your
baseUrl
tohttps://publishing-api.staging.publishing.service.gov.uk:8443
.Since the load testing is now limited by your SSH proxy, you may want to increase the
ulimit
to a high number such asulimit -n 4096
. This means SSH is able to cope with more open sockets.Steps:
- Put content
- Patch links
- Publish
-
govuk.WhitehallPublishing
Requires:
signonUrl
property.GATLING_USERNAME
andGATLING_PASSWORD
environment variables.Optional:
schedule
property will schedule publication.
This value must be a timestamp in the formatyyyy-MM-ddTHH:mm
(eg.2019-01-10T17:30
). The value must be at least 15 minutes before the test run as Whitehall enforces this rule for scheduled publishing.Optional:
maxTryAttempt
property will set how many times each step will be attempting before logging as a failure. This is to stimulate a user refreshing or reattempting after receiving an error response. If nothing is passed, it will default to 1.Example:
$ export JAVA_OPTS="-DbaseUrl=https://whitehall-admin.staging.publishing.service.gov.uk/ -Dworkers=1 -DsignonUrl=https://signon.staging.publishing.service.gov.uk/"
Steps:
- Authenticates with signon
- Drafts a publication
- Attaches an HTML attachment
- Tags to taxonomy
- Force publishes or force schedules
-
govuk.WhitehallPublishingCollections
Requires:
signonUrl
property.GATLING_USERNAME
andGATLING_PASSWORD
environment variables.Optional:
documentSearches
property - How many document searches to make when adding to the collection.Steps:
- Authenticates with signon
- Drafts a collection
- Searches for Gatling Test publications
- Adds search results to collection
- Tags to taxonomy
- Force publishes
-
govuk.Search
Data files: search.csv
Optional:
factor
(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits
, each worker requestsbase_path
ceil(hits * factor / workers)
times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration
(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time.If you are having difficulty running the entire test plan on a single machine within your desired duration, try splitting up the data file and running multiple instances of Gatling simultaneously on different machines.
5. Uploading
See how to upload results for more information on how to upload results so they are easy to access in the future.
6. Combining concurrent test results
It may be useful to run multiple concurrent Gatling tests to simulate more load than a single Gatling instance could generate. See how to combine results from concurrent tests.
7. Troubleshooting
-
My requests are being rate limited
Set the
rateLimitToken
property, and make sure the token is valid for the environment you’re testing. These tokens live in the encrypted hieradata in govuk-secrets. -
Authentication issues
- Ensure the right permissions have been set for your test user
- Disable 2FA
-
GATLING_USERNAME
needs to be an email address
-
No such file or directory
If you see an error such as:
15:16:48.524 [ERROR] i.g.a.Gatling$ - Run crashed java.io.FileNotFoundException: test-data/lorem-ipsum.txt (No such file or directory)
Ensure you run gatling in this repo,
govuk-load-testing
:export GATLING_HOME=/Users/username/gatling/ $GATLING_HOME/bin/gatling.sh
-
Exit scenario
This may be useful to remove workers that fail at any step:
.exec{ //CODE }.exitHereIfFailed
-
Output html
.exec(session => { val response = session("BODY").as[String] println(s"Response body: \n$response") session })