Skip to main content

Infrastructure

Last updated: 19 Apr 2021

Manage RabbitMQ

RabbitMQ is a message broker based on the Advanced Message Queuing Protocol (AMQP).

Learn more about RabbitMQ.

Connecting to the RabbitMQ web control panel

Run gds govuk connect rabbitmq -e integration aws/rabbitmq and point your browser at http://127.0.0.1:15672.

RabbitMQ metrics

A generic RabbitMQ dashboard shows metrics for queues and exchanges.

How we run RabbitMQ

Overview

A graph showing the message flow

Producer: an application that publishes messages RabbitMQ. On GOV.UK this could be publishing-api.

Exchanges are AMQP entities where messages are sent. They take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings. When a content change is made in a publishing application (e.g. Travel Advice Publisher), Publishing API publishes a message to our main exchange, publishing_documents.

Queues are very similar to queues in other message and task-queueing systems: they store messages that are consumed by applications. An example of a queue is email_alert_service which is used by email-alert-service to forward publishing activity to email-alert-api. Queues are created by consumer applications.

Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q, Q has to be bound to E. Bindings may have an optional routing key attribute. An example of a binding is the cache_clearing_service-high queue is bound to the published_documents exchange with a routing key matching of *.major. E.g messages sent to the exchange with a routing key of guide.major will be routed to that queue.

Messages consist of a JSON payload and publish options (we predominantly use content type, routing key and persistant).

Options:

  • content_type (string) - tells the consumer the type of message. E.g "application/json"
  • routing_key (string) - matches against bindings to filter messages to certain queues. E.g "guide.major"
  • persistant (boolean) - tells RabbitMQ whether to save the message to disk.

Message options are set when a message is published. In our use case, the message’s payload is the content item in JSON format. The code in the publishing-api to publish a message is here.

Consumer applications are applications which consume messages from one or more queues. For email-alert-service this is done by running this rake task and using the major change processor to do the processing of the consumed messages. All our consumer applications use the govuk_message_queue_consumer gem to consume messages from RabbitMQ in a standardised way.

You can see live examples of things like queues, exchanges, bindings etc by connecting to the RabbitMQ control panel.

Further reading