GOV.UK
Manual
Top tasks
- Deploy an application to GOV.UK
- Deploy emergency publishing banners
- Purge a page from cache
- Reboot a machine
- Run a rake task
- Useful Kibana queries
Manual
Expired pages
- MongoDB backups
- Technical setup
- Low available disk space
- AWS SES quota usage higher than expected
- MySQL: replication lag
- Prolonged GC collection times Check
- Offsite backups
- RabbitMQ: No consumers listening to queue
- Clone a MySQL instance from one slave to another
- Debian packaging
- Run a rake task
Pages due for review this week
- Fetch analytics data for search failed
- App isn't running the expected Ruby version
- Benchmark search queries failed
- Check for spelling suggestions failed
- Fix stuck virus scanning
- Rotate offsite backup GPG keys
Icinga alerts
- App healthcheck failed
- App isn't running the expected Ruby version
- Asset master attachment processing
- AWS SES quota usage higher than expected
- Benchmark search queries failed
- Check for spelling suggestions failed
- Check that correct users have access
- ClamAV definitions out of date
- Data sync
- Defined CPU type does not match
- duplicate SSH host keys
- Elasticsearch cluster health
- Elasticsearch: not receiving Syslog from Logstash
- Email alerts not sent
- es-rotate
- Fastly error rate for GOV.UK
- Fetch analytics data for search failed
- Fix stuck virus scanning
- Free memory warning on backend
- GitHub Enterprise connectivity
- Gor
- high disk time
- High memory for application
- high zombie procs
- Logs are not being received from the CDN
- Low available disk inodes
- Low available disk space
- mongod replication lag
- MongoDB rollback
- MySQL Xtrabackups to S3
- MySQL: replication lag
- MySQL: replication running
- Nginx 5xx rate too high for many apps/boxes
- Nginx high conn writing -- upstream indicator Check
- Nginx requests too low
- ntp drift too high
- Offsite backups
- Onsite backups
- Outstanding security updates
- PagerDuty drill
- Pingdom check for homepage failing
- Pingdom search check
- PostgreSQL: replication too far behind
- PostgreSQL: S3 Backups
- Prolonged GC collection times Check
- publisher app healthcheck not ok
- Puppet last run errors
- RabbitMQ: high watermark has been exceeded
- RabbitMQ: No consumers listening to queue
- reboot required by apt
- Redis alerts
- rkhunter warnings
- root filesystem is readonly
- Unicorn Herder
- Varnish port not responding
- VPN down
- Whitehall app healthcheck not ok
- Whitehall scheduled publishing
Assets
- Assets: how they work
- Remove an asset
- Rescan an "infected" file in Whitehall
- Upload an asset to asset-manager
Backups
- Attachment backups
- Backup and restore Elasticsearch indices
- MongoDB backups
- MySQL backups
- PostgreSQL backups
- Restore from offsite backups
- Rotate offsite backup GPG keys
CDN & Caching
DNS
Databases
- Clone a MySQL instance from one slave to another
- Replicating application data locally for development
- Resync a MongoDB database
- Set up PostgreSQL replication
- Upgrading MySQL
Deployment
- Block apps from being deployed
- Deploy an application to GOV.UK
- Deploy fixes for a security vulnerability
- Deploy Puppet
- Deploy when GitHub is unavailable
- Fall back to the static mirrors
- Handle encrypted hieradata
- Monitor your app during deployment
- Restart an application
- Run a rake task
- Set up Heroku review apps for pull requests
Environments
- Add a disk to a vCloud machine
- Create a new environment for GOV.UK
- Generate a Certificate Signing Request (CSR) for GOV.UK
- GOV.UK's environments (integration, staging, production)
- Reboot a machine
- Remove a machine
- Reprovision a machine in vCloud Director
- Set up a new mirror for GOV.UK
- Update Pingdom IP Ranges
Logging
Monitoring
Packaging
- Add a deployment dashboard for an application
- Create a new open source project
- Debian packaging
- Packer Vagrant Dev VM / Image Creation
- Publish to Puppet Forge
- Retire an application
- Set up a new Rails app
- Switch an app off
- Upgrading Rails
Publishing
Routing
- Archive and redirect mainstream content to other pages on GOV.UK
- Change an organisation's slug
- Check for a `gone` route
- Edit an existing route in the Router
- Redirect a route
Support
- Create a GPG key
- Manage sign on accounts
- Out of hours support (on-call)
- Technical setup
- Troubleshooting Vagrant
Testing
- Jenkins CI infrastructure
- Patching Jenkins
- Smoke testing with smokey
- Test & build a project on Jenkins CI
Tools
- Access apps on the shared Heroku account
- Add a new Ruby version
- GitHub Enterprise
- Graphite and deployment dashboards
- How we use GitHub
- Keeping software current
- Monitoring screens
- Pact Broker
- RabbitMQ
- Run an A/B test
- Sidekiq
- Tools: Icinga, Grafana and Graphite, Kibana and Fabric
- Update Hubot (Slack bot)