RabbitMQ is a message broker based on the Advanced Message Queuing Protocol (AMQP). Publishing’s RabbitMQ cluster is provided by AWS’ AmazonMQ service.
Connecting to the AmazonMQ web control panel
gds govuk connect amazonmq -e integration and point your
browser at the URL it gives you - it will look like http://127.0.0.1:45612, but will have a random port number. You can connect to
production the same way, just replace
integration above with the environment of your choice.
The username for connecting to the RabbitMQ web control panel is
root and the password
can be decrypted from the
govuk-secrets repo via
bundle exec rake 'eyaml:decrypt_value[integration,govuk_amazonmq::root_password]' (from the
A generic AmazonMQ dashboard shows metrics for queues and exchanges.
How we run RabbitMQ
Producer: an application that publishes messages to RabbitMQ. On GOV.UK this could be publishing-api.
Exchanges are AMQP entities where messages are sent. They take a message
and route it into zero or more queues. The routing algorithm used depends on
the exchange type and rules called bindings. When a content change is made
in a publishing application (e.g. Travel Advice Publisher), Publishing API
publishes a message to our main exchange,
Queues are very similar to queues in other message and task-queueing
systems: they store messages that are consumed by applications. An example of a
email_alert_service which is used by
email-alert-service to forward publishing activity to
email-alert-api. Queues are created by consumer applications.
Bindings are rules that exchanges use (among other things) to route
messages to queues. To instruct an exchange E to route messages to a queue Q, Q
has to be bound to E. Bindings may have an optional routing key attribute. An
example of a binding is the
cache_clearing_service-high queue is
bound to the
published_documents exchange with a routing
key matching of
*.major. E.g messages sent to the exchange with a routing key
guide.major will be routed to that queue.
Messages consist of a JSON payload and publish options (we predominantly use content type, routing key and persistant).
- content_type (string) - tells the consumer the type of message. E.g
- routing_key (string) - matches against bindings to filter messages to certain
- persistant (boolean) - tells RabbitMQ whether to save the message to disk.
Message options are set when a message is published. In our use case, the message’s payload is the content item in JSON format. The code in the publishing-api to publish a message is here.
Consumer applications are applications which consume messages from one or more queues. For email-alert-service this is done by running this rake task and using the major change processor to do the processing of the consumed messages. All our consumer applications use the govuk_message_queue_consumer gem to consume messages from RabbitMQ in a standardised way.
You can see live examples of things like queues, exchanges, bindings etc by connecting to the RabbitMQ control panel (see above).
- RabbitMQ Tutorials
- AmazonMQ documentation
- Bunny is the RabbitMQ client we use.
- The Bunny Guides explain all AMQP concepts really well.