GOV.UK and Virtual Private Networks (VPNs)
GOV.UK uses several VPNs to connect environments. This page explains what they are and what happens if they stop working.
VPN between live organisation and disaster recovery organisation
There's a VPN in each Carrenza environment (currently staging and production) which connects the live organisation to the disaster recovery organisation. It's used for monitoring the disaster recovery machines and for syncing data to them. The VPNs are managed by Carrenza.
If it goes down, these things will happen:
- The machines on the disaster recovery side of the connection will appear as unreachable hosts in GOV.UK monitoring
- Data replication will pause
VPN between AWS and UKCloud for Licensify Civica payment status requests
There's a VPN between AWS Production (only) and UKCloud Production which exists only as a workaround for routing certain requests from Licensify to Civica, one of Licensify's payment gateways.
If this VPN is down:
- The check_uk_cloud_vpn_up alert will fire in Icinga.
- Users who are paying for licence applications to certain licencing authorities will still be able to complete their application but the last step of their journey will display a message saying "We have received your application, but were unable to confirm payment with the authority." (source)
- The page still gives the user a reference number for their transaction and asks the user to contact the licencing authority to confirm that they have received the payment.
- Payments will still be processed as normal. The only difference is that Licensify is unable to tell the user whether the payment went through or not.
- Only those licencing authorities which use Civica as their payment processor are affected. This is a small but significant minority.
- Licencing authorities who do not use Civica are not affected.
Troubleshooting steps (aim is to switch off and on the VPN):
- Go to Production Skyscape portal, the credentials are in GOV.UK 2ndline Pass under:
- Once you logged in, you have to log into the
Productionorganization by selecting:
VMWARE CLOUDand then
GOV.UK Production. You will be asked for the password again.
- In the
Productionorganization, go to the
GOV.UK Managementvirtual datacenter.
edgesin the left column to bring the list of edges.
- Click on the
GOV.UK Managementedges in the right main frame and
Configure Servicesabove it.
- In the pop-up window, click on
VPNin the menu bar and then
IPsec VPN Sites
- In the list of VPN sites, select
UKC Licensify to AWSVPN and click on the edit icon above.
- In the new pop-up window, turn the VPN off by toggling the
enableswitch and clicking
- The pop-up window will disappear and you need to click
save changesin put into effect the VPN being now disabled.
- Wait a few minutes and repeat step 7-9 to re-enable the VPN again.
VPN between live organisation in Carrenza and AWS during AWS migration
During the staged migration of the GOV.UK from Carrenza to AWS, there is a VPN tunnel to connect the 2 private clouds so that migrated and un-migrated services can still access each other during the migration.
If the VPN goes down, these things will happen:
the ping probes between certain virtual machines in AWS and Carrenza will fail and these will appear in the Icinga dashboard of GOV.UK
services which makes use of the VPN will fail and the relevant alerts will appear in Icinga
Check that the VPN is
UPin the AWS Console by:
log in into the AWS console and assume your role for the given GOV.UK environment
go to the
VPCservice dashboard, as pictured in Figure 1.0 Figure 1.0: VPC dashboard in AWS showing the VPN site-to-site status.
scroll on the left pane of the dashboard and click
Site-to-Site VPN Connectionsas shown in step 1 in Figure 1.0
Staging AWSVPN connection as shown in step 2 in Figure 1.0
select the tab
Tunnel Detailsas shown in step 3 in Figure 1.0
check the status of the VPN tunnel, in the normal scenario, only one tunnel will be
UPas shown in step 4 in Figure 1.0. The other tunnel is
If all tunnels are down this means there is a VPN issue and you may have to swap to the secondary VPN tunnel.
Switching to the secondary VPN tunnel:
log in the Carrenza vCloud interface as described here
click on the "Administration" button on the interface as shown in step 1 in Figure 2.0 Figure 2.0: Carrenza vCloud interface showing the vEdge gateways list
GOV.UK Managementvirtual data centre and the data centre should appear as shown in step 2 in Figure 2.0
Edge Gatewaystab as shown in step 3 in Figure 2.0
right-click the only edge gateway in the list as shown in step 4 in Figure 2.0 and select
Edge Gateway Service.... A new pop-up window will appear as shown in Figure 2.1.
enabled(shown by a green tick in the enabled column) AWS VPN as shown in step 1 in Figure 2.1. Make a note of it so you remember which VPN you are going to disable. Do not select the UKCloud VPN
click on the
editbutton as shown in step 2 in Figure 2.1. A new pop-up window will appear where you can
Enable this VPN configurationoption so that this VPN is disabled. Finally select
okto return to the previous pop-up window.
in the list of VPN connections, select the second AWS VPN tunnel and click
editas in step 2 in Figure 2.1 and enable the VPN configuration in the new pop-up window.
finally click on
okas shown in step 3 in Figure 2.1.
go to the AWS console and check if the other VPN tunnel is now
UPand the previously active VPN tunnel is
In addition, check that the Icinga ping alerts across the Carrenza-AWS VPN has now been cleared.