Setup maintenance mode
What is maintenance mode?
Maintenance mode is a way of displaying a user-friendly message on a public facing application to inform them that a service is unavailable e.g. due to planned downtime. This is more helpful than routing them to an error page.
The example below uses the notice component to display the message.
Setup maintenance mode
There are three main steps involved:
- Create a maintenance flag
- Disable/Enable E2E(end-to-end) tests
- Enable/Disable the maintenance mode in govuk-helm-charts
1. Create a maintenance flag
Note: This is a one-time setup that is done in the application
Add the initializer file for e.g.
maintenance.rbto read the environment variables underconfig/initializers/Rails.application.config.maintenance_mode = ENV["MAINTENANCE_MESSAGE"].present? Rails.application.config.maintenance_message = ENV["MAINTENANCE_MESSAGE"]
In app view template
This is done when only a part of the application requires downtime.
- Identify the service that would be impacted
- Wrap the code to be shown/hidden in a conditional statement (e.g.
unless), and include the notice component which uses theRails.application.config.maintenance_message, see this commit as an example.
In routes file
When we need to ensure that all routes serve the maintenance page during downtime, we can enable the maintenance flag in routes.rb. This diverts all incoming traffic to a dedicated maintenance controller, regardless of the request method (GET, POST, etc.)
Add the below line in
routes.rbfile to redirect all incoming requests to theMaintenanceController:match "*" => "maintenance#show", via: %i[get post] if Rails.application.config.maintenance_modeAdd the notice component in a
maintenance viewpage
2. Disable/Enable E2E(end-to-end) tests
During planned outages, E2E (end-to-end) tests — if any — should be disabled to prevent CI/CD pipelines from failing or blocking deployments while the application is unavailable.
To disable a test from a specific environment add:
{ tag: ["@not-<environment"] }See govuk-e2e-tests as an example
Create/merge a revert PR to re-enable them once the system downtime is completed.
3. Enable/Disable the maintenance mode in govuk-helm-charts
Note: This is done right when you are about to make changes and the system becomes unavailable to users.
- Add the configuration for maintenance mode in
govuk-helm-chartsfor that particular environment like Integration, Staging or Production Update the maintenance message optionally with the downtime that the app might need to recover in
charts/app-config/<values-env>.yamlfile of the appropriate environment- name: MAINTENANCE_MESSAGE value: "The system will be unavailable for essential maintenance until 14:30, please try again later"If required, scale down any dependent applications and/or workers - not the app in maintenance mode - by setting
appEnabled: falseas in here. This prevents job processing while the app is in maintenance mode.Merge the changes
Navigate to Argo CD where we can see app-config update and apply the changes
The user facing page now displays the maintenance message!
Create and merge the Revert PR once the system changes are done so that the app renders the original content.