Repository: govuk_sidekiq

Gem to provide common config for Sidekiq in GOV.UK applications

• GitHub: govuk_sidekiq
• Ownership: #govuk-publishing-platform
• Hosting: N/A
• Category: Gems
• Links:
  • Kibana App Logs (Production, last 3 hours)
  • Kibana Worker Logs (Production, last 3 hours)

README

Provides a unified set of configurations and behaviours when using Sidekiq in GOV.UK applications.

Features

What does govuk_sidekiq do for you?

1. Makes sure Sidekiq can connect to Redis correctly, using the default environment variables (these are set in govuk-puppet).
2. Makes sure that the correct HTTP headers are passed on to gds-api-adapters. This means that for each request a unique ID (govuk_request_id) will be passed on to downstream applications. Read more about request tracing.
3. Makes sure that we use JSON logging, so that Sidekiq logs will end up properly in Logit and searchable through Kibana.

Installation (Rails only)

1. Add the gem

# Gemfile
gem "govuk_sidekiq"

2. Add a Sidekiq config file

# config/sidekiq.yml
---
:concurrency:  2

This file also allows you to configure queues with priority. See the Sidekiq wiki for available options.

3. Configure environment variables in EKS

For each environment (integration, staging and production), add a REDIS_URL environment variable for your application with a value of redis://shared-redis-govuk.eks.{environment}.govuk-internal.digital.

Additionally, set the value of workerEnabled to true for your application. This will result in a worker process running alongside the web application. The queue length and max delay can be monitored using the Sidekiq Grafana dashboard, once the Sidekiq worker is initialised.

There’s no step-by-step guide for this, but you can copy the changes made when Sidekiq was added to the release application. You may also want to resize resource requests for the app depending on the predicted request rate.

# your-application/config/deploy.rb
after "deploy:restart", "deploy:restart_procfile_worker"

4. Create some jobs

You can use normal Sidekiq jobs:

# app/workers/hard_worker.rb
class HardWorker
  include Sidekiq::Worker
  def perform(name, count)
    # do something
  end
end

Note that the convention is to use app/workers as the directory for your workers.

Testing

See Sidekiq testing documentation on how to test Sidekiq workers.

Because of the way we use middleware, you may see errors that indicate that your job is called with the wrong number of arguments. To set up testing correctly, replace require 'sidekiq/testing' with:

require 'govuk_sidekiq/testing'

Licence

MIT License