Skip to main content
Table of contents

Infrastructure

Manage RabbitMQ

RabbitMQ is a message broker based on the Advanced Message Queuing Protocol (AMQP).

Learn more about RabbitMQ.

Connecting to the RabbitMQ web control panel

Run gds govuk connect rabbitmq -e integration aws/rabbitmq and point your browser at http://127.0.0.1:15672.

RabbitMQ metrics

A generic RabbitMQ dashboard shows metrics for queues and exchanges.

How we run RabbitMQ

Overview

A graph showing the message flow

Producer: an application that publishes messages RabbitMQ. On GOV.UK this could be publishing-api.

Exchanges are AMQP entities where messages are sent. They take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings. When a content change is made in a publishing application (e.g. Travel Advice Publisher), Publishing API publishes a message to our main exchange, publishing_documents.

Queues are very similar to queues in other message and task-queueing systems: they store messages that are consumed by applications. An example of a queue is email_alert_service which is used by email-alert-service to forward publishing activity to email-alert-api. Queues are created by consumer applications.

Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q, Q has to be bound to E. Bindings may have an optional routing key attribute. An example of a binding is the cache_clearing_service-high queue is bound to the published_documents exchange with a routing key matching of *.major. E.g messages sent to the exchange with a routing key of guide.major will be routed to that queue.

Messages consist of a JSON payload and publish options (we predominantly use content type, routing key and persistant).

Options:

  • content_type (string) - tells the consumer the type of message. E.g "application/json"
  • routing_key (string) - matches against bindings to filter messages to certain queues. E.g "guide.major"
  • persistant (boolean) - tells RabbitMQ whether to save the message to disk.

Message options are set when a message is published. In our use case, the message's payload is the content item in JSON format. The code in the publishing-api to publish a message is here.

Consumer applications are applications which consume messages from one or more queues. For email-alert-service this is done by running this rake task and using the major change processor to do the processing of the consumed messages. All our consumer applications use the govuk_message_queue_consumer gem to consume messages from RabbitMQ in a standardised way.

You can see live examples of things like queues, exchanges, bindings etc by connecting to the RabbitMQ control panel.

Federated published_documents exchange

A federated exchange is connected to an upstream exchanges by AMQP and the messages published to the upstream exchanges are copied over to the federated exchange.

While the migration of GOV.UK from Carrenza to AWS is ongoing, we run two RabbitMQ clusters, one in each environment. Each cluster has a list of private IP addresses of the other cluster's nodes (Carrenza IPs and AWS IPs). The connection between Carrenza and AWS travels through the VPN. Since the nodes in AWS use dynamic IP addresses, they are associated to network interfaces with fixed IPs.

The published_documents exchange is federated in both directions, i.e. the RabbitMQ cluster from each provider connects as a client to the exchange in the other provider and forwards messages to its own exchange. There is no infinite loop because max-hops is set to 1.

Checking if the federation is ok

Connect to one of the cluster's nodes via:

$ gds govuk connect ssh -e production rabbitmq

and run:

$ sudo rabbitmqctl eval 'rabbit_federation_status:status().'

On one of the nodes, you should get a load of data including: {status,running}

If not, something is wrong and the federation is broken. Check the logs in /var/log/rabbitmq and verify that the credentials and IP address for the federation are correct, you can do this by running:

$ sudo rabbitmqctl list_parameters

Further reading

This page was last reviewed on 7 September 2020. It needs to be reviewed again on 7 March 2021 by the page owner #govuk-2ndline .
This page was set to be reviewed before 7 March 2021 by the page owner #govuk-2ndline. This might mean the content is out of date.