This page describes what to do in case of an
Icinga alert.
For more information you could
search the govuk-puppet repo for the source of the alert
Last updated: 22 Aug 2022
mongod replication lag
Investigating the problem
Begin by checking the Mongo cluster status, then checking the replication info
Possible fixes
Try restarting one of the lagging mongodb secondaries.
SSH into a non-primary machine and run:
sudo service mongodb restart
This may restart replication on that node, and also cause the other lagging node to resync with the primary node and restart its own replication.
If you need to restart mongodb on the other secondaries, do this one node at a time, as load on the primary mongo node may be increased by the replication.
If restarting doesn’t solve the problem, you can force a resync.