This repository contains the end-to-end (E2E) test suite for the GOV.UK platform.
This repository contains the end-to-end (E2E) test suite for GOV.UK, built using the Playwright framework.
Deployment & Operations
The test suite is automatically deployed to GOV.UK’s Kubernetes infrastructure on merge, just like any of our continuously deployed applications. Test traffic therefore originates from one of our three static NAT gateway IPs (there are no other identifiers, e.g. custom UA string, for test traffic at present).
The test suite is run regularly on weekdays only, as per the cron schedule.
Failures are not currently reported anywhere. Results can be seen in Argo.
Running the tests locally
Setup
Clone and navigate to the root folder.
Install the dependencies:
yarn install
yarn playwright install --with-deps chromium
Running a simple test locally
If you’re adding a simple new test and just want to check that it will pass, you can do the following:
- Comment out the authentication setup steps in
tests/auth.setup.js (lines 8-11)
- In the file that your test is in, comment out all the other tests so that your new ones are the only ones that will run.
- If your test has a relative path such as
page.goto("/contact/govuk");, change it to absolute an absolute path, for example: page.goto("https://www.gov.uk/contact/govuk");
- Run
npx playwright test but scope it to your test file. For example, if your tests are in feedback.spec.js you would run npx playwright test tests/feedback.spec.js
- If you want to test against integration, you can add the auth credentials directly in your
page.goto statement, for example await page.goto("https://yourusername:yourpassword@www.integration.publishing.service.gov.uk/contact/govuk");
Setting environment variables (for running the full test suite or more complex tests)
Create a .env file in the root of the project with the following content:
cat <<EOF > .env
PUBLISHING_DOMAIN=integration.publishing.service.gov.uk
PUBLIC_DOMAIN=www.integration.publishing.service.gov.uk
DGU_DOMAIN=www.integration.data.gov.uk
SIGNON_EMAIL=<email>
SIGNON_PASSWORD=<password>
EOF
Replace placeholders with appropriate values.
Run
yarn playwright test
Screenshot tests
By default, Playwright namespaces screenshots by operating system architecture (e.g. -darwin.png if running on a Mac, or -linux.png on Linux.) This would mean we would have to maintain multiple different screenshots. Therefore we have modified playwright.config.js to just maintain one version of each screenshot. Screenshot tests currently only run against Google Chrome as well (you can see this in the projects section of playwright.config.js). This greatly simplifies the implementation, as we only need to maintain a screenshot of the expected visual design for Chrome. Adding other browsers may be complicated, as they render our page with subtle pixel differences which causes the tests to fail. For example, the difference between our footer in Chrome and Firefox is as varied as 1500 pixels. Also, maintaining multiple browser screenshots would likely involve reenabling the operating system specific screenshots too, meaning we’d have to maintain -darwin.png screenshots to make tests run on local machines, and -linux.png screenshots to make the tests run on the server.
See the Playwright docs for more information: https://playwright.dev/docs/test-snapshots