Repository: govuk_sidekiq
Gem to provide common config for Sidekiq in GOV.UK applications
- GitHub
- govuk_sidekiq
- Ownership
- #govuk-publishing-platform
- Hosting
- N/A
- Category
- Gems
README
Provides a unified set of configurations and behaviours when using Sidekiq in GOV.UK applications.
Features
What does govuk_sidekiq
do for you?
- Makes sure Sidekiq can connect to Redis correctly, using the default environment variables (these are set in govuk-puppet).
- Makes sure that the correct HTTP headers are passed on to gds-api-adapters.
This means that for each request a unique ID (
govuk_request_id
) will be passed on to downstream applications. Read more about request tracing. - Makes sure that we use JSON logging, so that Sidekiq logs will end up properly in Kibana.
- Sends activity stats to Statsd, so that you can make pretty graphs of activity in Grafana or Graphite. See the Rummager dashboards for an example.
Installation (Rails only)
1. Add the gem
# Gemfile
gem "govuk_sidekiq"
2. Add a Sidekiq config file
# config/sidekiq.yml
---
:concurrency: 2
This file also allows you to configure queues with priority. See the Sidekiq wiki for available options.
3. Add a Procfile
This is what puppet uses to create the process.
# Procfile
worker: bundle exec sidekiq -C ./config/sidekiq.yml
4. Configure puppet
-
Set a
REDIS_URL
environment variable.GOVUK_APP_NAME
should also be set, but this is already done by the defaultgovuk::app::config
.Apply redis variables for your app in the default config. For example:
govuk::apps::your_app::redis_host: "%{hiera('sidekiq_host')}" govuk::apps::your_app::redis_port: "%{hiera('sidekiq_port')}"
-
Make sure puppet creates and starts the Procfile worker.
There’s no step-by-step guide for this, but you can copy the config from collections-publisher.
5. Configure deployment scripts
Make sure you restart the worker after deploying by adding a hook to the capistrano scripts in govuk-app-deployment. Otherwise the worker will keep running old code.
# your-application/config/deploy.rb
after "deploy:restart", "deploy:restart_procfile_worker"
6. Add app to sidekiq-monitoring
See the dev docs for a step-by-step guide: Add sidekiq-monitoring to your application
8. Create some jobs
You can use normal Sidekiq jobs:
# app/workers/hard_worker.rb
class HardWorker
include Sidekiq::Worker
def perform(name, count)
# do something
end
end
Note that the convention is to use app/workers
as the directory for your workers.
Testing
See Sidekiq testing documentation on how to test Sidekiq workers.
Because of the way we use middleware, you may see errors that indicate that
your job is called with the wrong number of arguments. To set up testing
correctly, replace require 'sidekiq/testing'
with:
require 'govuk_sidekiq/testing'