Manage RabbitMQ
RabbitMQ is a message broker based on the Advanced Message Queuing Protocol (AMQP).
Connecting to the RabbitMQ web control panel
Run gds govuk connect rabbitmq -e integration aws/rabbitmq and point your
browser at http://127.0.0.1:15672 (also available for staging and production).
The username for connecting to the RabbitMQ web control panel is root and the password
can be decrypted from the govuk-secrets repo via bundle exec rake 'eyaml:decrypt_value[integration,govuk_rabbitmq::root_password]' (from the puppet_aws directory).
RabbitMQ metrics
A generic RabbitMQ dashboard shows metrics for queues and exchanges.
How we run RabbitMQ
Overview
Producer: an application that publishes messages to RabbitMQ. On GOV.UK this could be seed-crawler.
Exchanges are AMQP entities where messages are sent. They take a message
and route it into zero or more queues. The routing algorithm used depends on
the exchange type and rules called bindings. Seed Crawler publishes a message
for every line in the GOV.UK sitemap, to the exchange
govuk_crawler_exchange.
Queues are very similar to queues in other message and task-queueing
systems: they store messages that are consumed by applications. An example of a
queue is govuk_crawler_queue which is used by
[govuk_crawler_worker][https://github.com/alphagov/govuk_crawler_worker] as a list
of URLs to process. Queues are created by consumer applications.
Bindings are rules that exchanges use (among other things) to route
messages to queues. To instruct an exchange E to route messages to a queue Q, Q
has to be bound to E. Bindings may have an optional routing key attribute. An
example of a binding is the cache_clearing_service-high queue is
bound to the published_documents exchange with a routing
key matching of *.major. E.g messages sent to the exchange with a routing key
of guide.major will be routed to that queue.
Messages consist of a JSON payload and publish options (we predominantly use content type, routing key and persistant).
Options:
- content_type (string) - tells the consumer the type of message. E.g
"application/json" - routing_key (string) - matches against bindings to filter messages to certain
queues. E.g
"guide.major" - persistant (boolean) - tells RabbitMQ whether to save the message to disk.
Message options are set when a message is published. In our use case, the message’s payload is the content item in JSON format. The code in the publishing-api to publish a message is here.
Consumer applications are applications which consume messages from one or more queues. For email-alert-service this is done by running this rake task and using the major change processor to do the processing of the consumed messages. All our consumer applications use the govuk_message_queue_consumer gem to consume messages from RabbitMQ in a standardised way.
You can see live examples of things like queues, exchanges, bindings etc by connecting to the RabbitMQ control panel.
Further reading
- RabbitMQ Tutorials
- Bunny is the RabbitMQ client we use.
- The Bunny Guides explain all AMQP concepts really well.