Connecting to the RabbitMQ web control panel
gds govuk connect rabbitmq -e integration aws/rabbitmq and point your
browser at http://127.0.0.1:15672.
A generic RabbitMQ dashboard shows metrics for queues and exchanges.
How we run RabbitMQ
Producer: an application that publishes messages RabbitMQ. On GOV.UK this could be publishing-api.
Exchanges are AMQP entities where messages are sent. They take a message
and route it into zero or more queues. The routing algorithm used depends on
the exchange type and rules called bindings. When a content change is made
in a publishing application (e.g. Travel Advice Publisher), Publishing API
publishes a message to our main exchange,
Queues are very similar to queues in other message and task-queueing
systems: they store messages that are consumed by applications. An example of a
email_alert_service which is used by
email-alert-service to forward publishing activity to
email-alert-api. Queues are created by consumer applications.
Bindings are rules that exchanges use (among other things) to route
messages to queues. To instruct an exchange E to route messages to a queue Q, Q
has to be bound to E. Bindings may have an optional routing key attribute. An
example of a binding is the
cache_clearing_service-high queue is
bound to the
published_documents exchange with a routing
key matching of
*.major. E.g messages sent to the exchange with a routing key
guide.major will be routed to that queue.
Messages consist of a JSON payload and publish options (we predominantly use content type, routing key and persistant).
- content_type (string) - tells the consumer the type of message. E.g
- routing_key (string) - matches against bindings to filter messages to certain
- persistant (boolean) - tells RabbitMQ whether to save the message to disk.
Message options are set when a message is published. In our use case, the message's payload is the content item in JSON format. The code in the publishing-api to publish a message is here.
Consumer applications are applications which consume messages from one or more queues. For email-alert-service this is done by running this rake task and using the major change processor to do the processing of the consumed messages. All our consumer applications use the govuk_message_queue_consumer gem to consume messages from RabbitMQ in a standardised way.
You can see live examples of things like queues, exchanges, bindings etc by connecting to the RabbitMQ control panel.
A federated exchange is connected to an upstream exchanges by AMQP and the messages published to the upstream exchanges are copied over to the federated exchange.
While the migration of GOV.UK from Carrenza to AWS is ongoing, we run two RabbitMQ clusters, one in each environment. Each cluster has a list of private IP addresses of the other cluster's nodes (Carrenza IPs and AWS IPs). The connection between Carrenza and AWS travels through the VPN. Since the nodes in AWS use dynamic IP addresses, they are associated to network interfaces with fixed IPs.
published_documents exchange is federated in both
directions, i.e. the RabbitMQ cluster from each provider connects as a client
to the exchange in the other provider and forwards messages to its own
exchange. There is no infinite loop because
max-hops is set to
Checking if the federation is ok
Connect to one of the cluster's nodes via:
$ gds govuk connect ssh -e production rabbitmq
$ sudo rabbitmqctl eval 'rabbit_federation_status:status().'
On one of the nodes, you should get a load of data including:
If not, something is wrong and the federation is broken. Check the logs in
/var/log/rabbitmq and verify that the credentials and IP address for the
federation are correct, you can do this by running:
$ sudo rabbitmqctl list_parameters