Get started on GOV.UK
This is the guide for new technical staff working on GOV.UK in GDS. If you just joined, 👋 welcome!
If they haven’t done so yet, ask your tech lead to take you through the overview slides.
Note that if you’re not working for GDS you’ll not be able to complete all of the steps in this guide.
You’ll need to use a Mac to follow this guide.
It will take you roughly one day to do everything in this guide from start to finish. There are lots of things to download, and loads of installers need to do their thing.
You’ll need up to 150GB of free space on your hard-drive to run the whole of GOV.UK.
Developing outside the VM
You don’t have to develop on the VM, but we strongly recommend it. If you have problems with the development VM you can always ask for help in the #govuk-developers Slack channel.
Where you should run commands
Run commands prepended with
mac$ on your Mac:
mac$ echo "Think Different"
Run commands prepended with
dev$ in the development VM:
dev$ echo "Linux for human beings"
mac$are prompts within this guide and not part of the given command.
If you run into problems
If you’re having trouble with Vagrant or the development VM, you can ask your colleagues or the #govuk-developers channel in Slack.
1. Install some dependencies
- git command line tool: either from the Managed Software Center or from git-scm
- The vagrant-dns plugin (
vagrant plugin install vagrant-dns)
- rbenv (via Homebrew -
brew install rbenv- you can install Homebrew via the Managed Software Center)
Starting with High Sierra 10.13, kernel extensions must be approved by the user (see this Apple technical note). This causes the VirtualBox installer to fail with a permissions error.
To install VirtualBox on High Sierra 10.13 or later:
- Run the VirtualBox installer
- Open “Security & Privacy” in the system preferences
- Allow the blocked VirtualBox kernel extension
- Run the VirtualBox installer again
If you have trouble installing the Vagrant DNS plugin due to Xcode version compatibility issues, you can try installing an older version of the plugin:
mac$ vagrant plugin install vagrant-dns --plugin-version 1.1.0
2. Create your GitHub account
- Set up a GitHub account.
- Ask somebody with access to add your GitHub username to the user monitoring system.
- Ask your tech lead to add you to the alphagov organisation. You will have to be added to the GOV.UK team to get access to repos & CI.
- Generate and register an SSH key pair for your Mac for your GitHub account.
Import the SSH key into your keychain. Once you’ve done this, it’ll be available to the VM you’ll install in the next step.
mac$ /usr/bin/ssh-add -K your-private-key
Test that it all works by running
ssh -T firstname.lastname@example.org.
3. Set up your local workspace
Some processes are dependent on your local gov.uk setup being in
mac$ mkdir ~/govuk mac$ cd ~/govuk mac$ git clone email@example.com:alphagov/govuk-puppet.git
3.1 Ensure your Ruby setup works
You’ll want to install the [specific version of Ruby in govuk-puppet]https://github.com/alphagov/govuk-puppet/blob/master/.ruby-version#L1, as the default system Ruby is too locked-down.
Now verify that rbenv is working as expected:
mac$ cd ~/govuk/govuk-puppet mac$ rbenv versions
This should output something like:
system * 1.9.3-p550 (set by /Users/yourname/govuk/govuk-puppet/.ruby-version)
mac$ which ruby
This should output something like:
If, instead, this outputs
/usr/bin/ruby, then you’ll need to update your
~/.bash_profile to have
ruby properly overridden in your
# force load rbenv to ensure correct version of ruby used export PATH="$PATH:~/.rbenv/shims" if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi
This Ruby setup is a pre-requisite for the “Import production data” step.
4. Boot your VM
Run the VM bootstrap script:
mac$ cd ~/govuk/govuk-puppet/development-vm mac$ vagrant up mac$ vagrant dns --install
This will take a little while, but it will throw up a question or two in your console so check back on it occasionally. Now might be a good time to scan through the GOV.UK technology blog while Puppet runs.
Once your VM is running, you can SSH into it with:
mac$ vagrant ssh
See the full command list by typing
Set your Git username and email
You can assign your name and email to commits on the VM:
dev$ git config --global user.email "firstname.lastname@example.org" dev$ git config --global user.name "Friendly Giraffe"
Config for ZSH users
If you use ZSH as your shell, you may find that the Ruby version on your VM is stuck on
1.9.3 and there seems to not be any other versions installed when you run
rbenv versions. To fix this, add
. /etc/profile to your
5. Set up your apps
Begin by checking out all of the GOV.UK services. There’s a handy shortcut:
dev$ cd /var/govuk/govuk-puppet/development-vm dev$ ./checkout-repos.sh < alphagov_repos
Most of our apps are written in Ruby and use Bundler to manage their dependencies. To boot apps, you’ll also need to install those dependencies:
There are also some Python apps, which use PIP. You’ll probably need to install those dependencies too, so run:
If installing the Python dependencies for fabric-scripts fails, your version of setuptools may be too old:
dev$ cd /var/govuk/fabric-scripts dev$ virtualenv .venv dev$ source .venv/bin/activate dev$ pip install --upgrade setuptools dev$ pip install -r requirements.txt
~/govuk/on your host machine is mounted as
/var/govukinside the VM. Any app repositories you clone should go here.
6. Get AWS access
👉 First, set up your AWS account
👉 Then, set up your CLI access for AWS
7. Import production data
Application data from production can be imported to be used within your development environment. Dumps of production data are generated in the early hours each day and made available from S3.
Download the data by running:
mac$ cd ~/govuk/govuk-puppet/development-vm/replication mac$ gem install bundler -v 1.17 # specific version as modern bundler requires ruby >= 2.3.0 mac$ bundle install mac$ ./replicate-data-local.sh -n
If you’ve set up your AWS account correctly, you should then be prompted to enter your MFA token.
Once you have downloaded or obtained the data, run the following to load the data into your databases:
dev$ cd /var/govuk/govuk-puppet/development-vm/replication dev$ ./replicate-data-local.sh -d backups/YYYY-MM-DD/ -s
Downloading and decompressing the data may take a long time. Depending on your application you may not need all the data. Run
./replicate-data-local.sh --helpfor options to skip databases.
For more information or troubleshooting advice see the guide in the developer docs on replicating application data locally for development.
8. Run your apps
You can run any of the GOV.UK apps from the
/var/govuk/govuk-puppet/development-vm directory. You’ll first need to run
bundle install in this folder to install the required gems.
Since many of our apps depend on other apps, we normally run them using bowler instead of foreman.
To run particular apps with bowler, use:
dev$ bowl content-tagger
This will also run all of the dependencies defined in the
If you don’t need an optional dependency, you can pass the
dev$ bowl whitehall -w mapit
bowl commands fail, try the troubleshooting guide on how to fix a broken bowl.
Now visit this URL once the app is running:
You should be able to see Whitehall.
9. Access the web frontend 🚀
Most GOV.UK web applications and services are available via the public internet, on the following forms of URL:
- http://publisher.dev.gov.uk (local dev, requires the application to be running)
- https://www-origin.integration.publishing.service.gov.uk (integration, HTTP basic auth)
- https://deploy.staging.publishing.service.gov.uk (staging, restricted to GDS office IP addresses)
- https://alert.publishing.service.gov.uk (production, restricted to GDS office IP addresses)
The basic authentication username and password is widely known, so just ask somebody on your team if you don’t know it.
10. Keep your VM up to date
There are a few scripts that should be run regularly to keep your VM up to date. In
govuk-puppet/development-vm there is
update-bundler.sh to help with this. Also,
govuk_puppet should be run from anywhere on the VM regularly.
The following script will do all of this for you.
dev$ cd /var/govuk/govuk-puppet/development-vm dev$ ./update-all.sh
This will run:
git pullon each of the applications checked out in
govuk_puppetto bring the latest configuration to the dev VM
bundle installfor each Ruby application to install any missing gems
pip installto update runtime dependencies for any Python apps