Skip to main content
Table of contents

Emails

Load Test Email Alert API

You may wish to load test email-alert-api to get a realistic idea of how the system performs under high load, or if your’re making changes to how emails are processed to ensure your changes have the desired effect.

You can load test email-alert-api with the staging version of Notify so that it won’t put unnecessary load on their production version. Let them know if you’re planning on doing a large scale test (anything more than a few thousand emails) and they can scale it up as necessary.

Setting up for the test

  1. Post on #2nd-line channel that you’re about to do some load testing and that as part of that you’ll be disabling puppet runs on staging for email-alert-api.
  2. Use a fabric script to disable puppet runs for email-alert-api so it won’t overwrite or remove the environment variables you need to set to run the test.
  3. If you’re running the test against Notify’s staging environment, you need to set a few environment variables. The API key is stored in the pass store in govuk-secrets under govuk-notify/test-api-key
Key Value Reason
GOVUK_NOTIFY_BASE_URL https://api.staging-notify.works/ We need to send requests to the correct endpoint
GOVUK_NOTIFY_API_KEY Key from govuk-secrets The production key will not be correct for staging
GOVUK_NOTIFY_TEMPLATE_ID 76d21ce7-54c3-4fb7-8830-ba3b79287985 Requests will fail if it cannot find the template, this exists only on staging
  1. You need to set the following environment variable key value pairs whether or not you’re sending emails to Notify staging:
Key Value Reason
EMAIL_ADDRESS_OVERRIDE success@simulator.amazonses.com Sends all emails to Amazon SES, preventing it going to real users
  1. Use fabric to restart the workers so they’ll pick up the new environment variables by running fab aws_staging node_type:email_alert_api app.restart:email-alert-api-procfile-worker
  2. Now you’re ready to start, open a rails console on an email-alert-api machine on staging. How you perform the test may depend on what you want to test, in the past we’ve duplicated content changes from a few hours or a day.

During the test

You can keep an eye on the Grafana dashboard to see how it’s performing. If it’s a large test, a lot of logs can be produced in a short amount of time so use the machine dashboard filtered by email-alert-api machines to check disk space usage.

After the test

Once you’ve finished the test and the queues have been cleared, re-enable puppet runs, run puppet in order to overwrite the environment variables you’ve set, restart the workers again and let 2nd line and Notify know you’ve finished.

This page was last reviewed on 13 January 2020. It needs to be reviewed again on 13 January 2021 by the page owner #govuk-platform-health .
This page was set to be reviewed before 13 January 2021 by the page owner #govuk-platform-health. This might mean the content is out of date.