Travel Advice or Drug and Medical Device email alerts not sent

This page describes what to do in case of an Icinga alert. For more information you could search the govuk-puppet repo for the source of the alert

Last updated: 11 Dec 2020

We expect that when new GOV.UK content is published, or when existing content is updated in a significant way, emails will be sent to any subscribers to notify them of the change.

We actively monitor this for medical safety alerts and travel advice updates with the Medical safety alerts check and Travel advice alerts check, these are configured in email-alert-monitoring.

These checks determine a list of content that we expect subscribers to have been emailed. We use a Gmail account which is subscribed to all travel advice and medical safety alerts to check whether an email has been received within a sufficient time period, if not the checks will fail.

Determining why the check failed

Use the "Console Output" of the Jenkins job for this check to determine the reason the job failed.

Assuming it has failed due to not being able to find emails there should be a list of email subjects that were expected to exist.

Verifying whether the email was received

To verify if the Gmail account has received the email or not you can log into the Gmail account, govuk_email_check@digital.cabinet-office.gov.uk, using the credentials in the 2nd line password store under google-accounts/govuk_email_check@digital.cabinet-office.gov.uk. Then you can use the previously noted subjects to try identify the email(s).

This check has been susceptible to a number of false positive scenarios in the past (there are plans to retire this check due, in part, to it's fragility), examples of these are:

the email contents didn't exactly match what was searched for due to the content being edited, these can be resolved by updating the acknowledged email list and re-running the Jenkins job;
Gmail decided the email was spam;
Gmail experienced service problems which delayed email receiving (status dashboard).

Troubleshooting emails that were not received

If you have verified that the email hasn't been received you should then investigate was the email sent. Some avenues to explore are:

determine whether Email Alert API has a backlog of work to do using the Sidekiq dashboard and Email Alert API Technical dashboard - the email may just be delayed;
whether the change was a result of a new type of medical safety content sub-type, this was the previous cause of an incident and is recorded as GOV.UK Tech Debt;
check whether the email was sent as a courtesy copy to determine if Email Alert API processed the change;
verify whether Notify sent the expected email using the support:view_emails rake task;
check the Kibana logs and Sentry for any errors or clues;
if all else fails you may need to investigate the Email Alert API database to determine whether the content change was received and what state it is in.

Resending travel advice emails

If you need to force the sending of a travel advice email alert, there is a rake task in Travel Advice Publisher, which you can run using this Jenkins job where the edition ID of the travel advice content item can be found in the URL of the country's edit page in Travel Advice Publisher and looks like fedc13e231ccd7d63e1abf65.