Skip to main content
Warning This document has not been updated for a while now. It may be out of date.
Last updated: 3 Jul 2024

Manage Amazon MQ

RabbitMQ is a message broker based on the Advanced Message Queuing Protocol (AMQP). Publishing’s RabbitMQ cluster is hosted using Amazon MQ.

Learn more about RabbitMQ.

Connect to the RabbitMQ web control panel

  1. Install the krelay kubectl plugin.

    brew install knight42/tap/krelay

  2. Obtain admin credentials to access the GOV.UK EKS Cluster

    # Obtain IAM credentials for the AWS account (integration, staging, production).
eval $(gds aws govuk-<environment>-admin -e --art 8h)

# Select the corresponding kubectl context (or create a new admin context if needed)
kubectl config use-context <your-admin-context-name>

  3. Forward the RabbitMQ HTTPS port to your local machine.

    k relay host/publishingmq.<environment>.govuk-internal.digital 4430:443

  4. Open https://localhost:4430/ in your browser. The TLS certificate will not match localhost, so navigate past the certificate warnings. In Chrome, you can set chrome://flags/#allow-insecure-localhost if you prefer.

  5. Log into the RabbitMQ Management web interface. The username is root and the passwords for each environment are in 2ndline/publishing-amazonmq in Secrets Manager in the production AWS account.

Amazon MQ metrics

A generic Amazon MQ dashboard shows metrics for queues and exchanges.

How we run RabbitMQ

Overview

A graph showing the message flow

Producer: an application that publishes messages to RabbitMQ. On GOV.UK this could be publishing-api.

Exchanges are AMQP entities where messages are sent. They take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings. When a content change is made in a publishing application (e.g. Travel Advice Publisher), Publishing API publishes a message to our main exchange, publishing_documents.

Queues are very similar to queues in other message and task-queueing systems: they store messages that are consumed by applications. An example of a queue is email_alert_service which is used by email-alert-service to forward publishing activity to email-alert-api. Queues are created by consumer applications.

Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q, Q has to be bound to E. Bindings may have an optional routing key attribute. An example of a binding is the email_alert_service queue is bound to the published_documents exchange with a routing key matching of *.major. E.g messages sent to the exchange with a routing key of guide.major will be routed to that queue.

Messages consist of a JSON payload and publish options (we predominantly use content type, routing key and persistant).

Options:

  • content_type (string) - tells the consumer the type of message. E.g "application/json"
  • routing_key (string) - matches against bindings to filter messages to certain queues. E.g "guide.major"
  • persistant (boolean) - tells RabbitMQ whether to save the message to disk.

Message options are set when a message is published. In our use case, the message’s payload is the content item in JSON format. The code in the publishing-api to publish a message is here.

Consumer applications are applications which consume messages from one or more queues. For email-alert-service this is done by running this rake task and using the major change processor to do the processing of the consumed messages. All our consumer applications use the govuk_message_queue_consumer gem to consume messages from RabbitMQ in a standardised way.

You can see live examples of things like queues, exchanges, bindings etc by connecting to the RabbitMQ control panel (see above).

Further reading

More in the Infrastructure section

Learn

How to...