Deploy an application to GOV.UK
2nd line is responsible for:
- ensuring that software is released to GOV.UK responsibly
- providing access to deploy software for teams who can’t deploy it themselves
As far as possible, teams are responsible for deploying their own work. We believe that regular releases minimise the risk of major problems and improve recovery time.
If possible, you should avoid coupling multiple applications so that they all have to be deployed at once. Changes should be backwards compatible except in exceptional circumstances.
We have a staging environment that must always be deployed to immediately before production. Access to staging and production is restricted to certain people.
Only one release takes place at a time. One product team owns the release - if multiple teams are involved in the release, pick one. Releases are tracked by the release app.
You need to have a Signon account with appropriate permissions to access the release app.
Deployment communications are in the
#govuk-deploy Slack channel. If you are
on 2ndline you should add yourself to that channel. Releases can
start from 9:30am and must be finished by 5pm, or 4pm on Fridays.
This process is new as of 15th May 2017, for more information about why we changed it and what the old way was, you can read RFC 70.
- Check with anyone whose changes you will release during your deploy (check the release app)
#govuk-deployrecent history and the channel topic
- Announce your deployment if it’s potentially problematic
- Deploy to staging
- Check Smokey passes; remember most apps take a couple of minutes to restart
- Check the new functionality works as you would expect
- Deploy to production
- Check Smokey passes in production; once again, apps take a couple of minutes to restart
- Check the new functionality works in production
- Take a look at any alerts and metrics, just to check you haven’t broken something
- Stay around for a while just in case something goes wrong
An alert for the start and end of your deployment will appear in the channel. Jenkins will still enforce sequential deployments per environment across all applications, so you may end up in a queue.
Holding deployment of applications
If you need to hold deployments of applications, you must add a note to the release app. Do this by clicking the edit button in the Notes column in the entry for that application.
2nd line support
If you need support from 2nd line during your deploy, contact them in advance and agree a time.
If you are responding to a security incident, follow the steps to deploy fixes for a security vulnerability.
Make sure you have a rollback plan if things go wrong. When you’re just changing code, this is relatively easy; when you’re doing data migrations, less so. As far as is possible, all data migrations should be reversible. Don’t rely on backups unless you absolutely have to.
If a release fails in staging and it can be easily and quickly fixed, you don’t
need to rollback the change. Just post a message in the
channel to let people know what’s going. Then test your fix in integration and
redeploy and test it in staging before continuing the deploy to production.
If a release fails in staging and it is not a quick-fix, you should rollback the change and try again later. Post a note in the release app saying not to deploy the application in the meantime.
We depend on GitHub for deploying software to GOV.UK. We have processes in place to deploy if GitHub is unavailable.
On the blog
- Releasing applications to GOV.UK (older post, using the badger)
More about Deployment
- Add a deployment dashboard for an application
- Block apps from being deployed
- Deploy fixes for a security vulnerability
- Deploy Puppet
- Deploy when GitHub is unavailable
- Fall back to the static mirrors
- Get access to Jenkins
- Handle encrypted hieradata
- Monitor your app during deployment
- Restart an application
- Retire an application
- Run a rake task
- Set up a new Rails app
- Set up Heroku review apps for pull requests
- Switch an app off