Last updated: 5 Mar 2024

Kubernetes/EKS cheatsheet for GOV.UK developers

This page lists some example commands for interacting with the GOV.UK Kubernetes clusters.

See https://kubernetes.io/docs/reference/kubectl/quick-reference/ for a more general quick-reference guide.

Prerequisites

1. You need to have completed Access an EKS cluster from the get started guide. If you skimmed those instructions, make sure you have configured your shell:

   export AWS_REGION=eu-west-1
   alias k=kubectl
   

2. Obtain credentials to access the cluster. Use an IAM role with sufficient permissions:

   • -readonly roles can view logs and configuration
   • -poweruser roles can run Rake tasks or open a shell
   • -administrator roles can modify base cluster services (you should not normally need this)

   For example:

   # Obtain IAM credentials for the AWS account (integration, staging, production).
   eval $(gds aws govuk-integration-poweruser -e --art 8h)
   
   # Select the corresponding kubectl context. `k config get-contexts` lists them.
   k config use-context integration
   
   # Use the `apps` namespace by default. You only need to do this the first
   # time you use each cluster.
   k config set-context --current --namespace=apps
   

Common tasks

View recent logs for an app

From one pod, chosen arbitrarily:

k logs deploy/account-api

From all pods:

k logs -l app=account-api

Open a Rails console

k exec -it deploy/router-api -- rails c

Open a shell in an app container

k exec -it deploy/government-frontend -- bash

Open a shell on Router

k exec -it deploy/router -c nginx

List the available Rake tasks for an app

k exec deploy/email-alert-api -- rake -T

Run a Rake task

k exec deploy/email-alert-api -- \
  rake 'support:view_emails[your.email@digital.cabinet-office.gov.uk]'

See Run a Rake task on EKS.

App releases and rollouts / deployments

• The Release app shows what’s deployed where.
• Deployment automation is via Argo CD. Post-deploy smoke tests run via Argo Workflows.
• To deploy a branch or manually promote a release:
  1. Go to the app’s repo in GitHub.
  2. Choose Actions.
  3. Choose Deploy from the left-hand column.
  4. Click Run workflow.
  5. Leave the Use workflow from dropdown set to main, unless you are testing a change to the workflow itself.
  6. Under Commit, tag or branch name to deploy, enter the branch or release tag.

Smokey test logs

Argo Workflows keeps logs only for failed workflow tasks.

The Smokey cronjob keeps the last 3 successes and the last 3 failures.

Dashboards

Dashboards are on Grafana.

You can see Sidekiq queue lengths on the Sidekiq Grafana dashboard.