Skip to main content
Table of contents
This page describes what to do in case of an Icinga alert. For more information you could search the govuk-puppet repo for the source of the alert

Icinga alerts

content-data-api app healthcheck not ok

If there is a health check error showing for Content Data API, you can click on the alert to find out more details about what’s wrong.

Note that:

  • The ETL process runs at 7am (UK time) in production.
  • The ETL process runs at 11am (UK time) in staging.
  • The ETL process runs at 1pm (UK time) in integration.
  • All dates for the rake tasks below are inclusive. In other words, if you only need to reprocess data for a specific day, you’ll need to use the same the date for both the ‘from’ and 'to’ parameters (for example: etl:repopulate_aggregations_month["2019-12-15","2019-12-15"]).
  • The rake task should be run on the content-data-api TARGET_APPLICATION and the backend MACHINE_CLASS.

Here are the possible problems you may see:

ETL :: no monthly aggregations of metrics for yesterday

This means that the ETL master process that runs daily that creates aggregations of the metrics failed.

To fix this problem run the following rake task:

etl:repopulate_aggregations_month["YYYY-MM-DD","YYYY-MM-DD"]

ETL :: no searches updated from yesterday

This means that the Etl process that runs daily and refreshes the Materialized Views failed to update those views.

To fix this problem run the following rake task:

etl:repopulate_aggregations_search

ETL :: no daily metrics for yesterday

This means that the ETL master process that runs daily to retrieve metrics for content items has failed.

To fix this problem re-run the master process again

Note This will first delete any metrics that had been successfully retrieved before re-running the task to regather all metrics.

ETL :: no pviews for yesterday

This means the the ETL master process that runs daily has failed to collect pageview metrics from Google Analytics. The issue may originate from the ETL processor responsible for collecting core metrics.

To fix this problem run the following rake task:

rake etl:repopulateviews["YYYY-MM-DD","YYYY-MM-DD"]

ETL :: no upviews for yesterday

This means the the ETL master process that runs daily has failed to collect unique pageview metrics from Google Analytics. The issue may originate from the ETL processor responsible for collecting core metrics.

To fix this problem run the following rake task:

rake etl:repopulateviews["YYYY-MM-DD","YYYY-MM-DD"]

ETL :: no searches for yesterday

This means the the ETL master process that runs daily has failed to collect number of searches metrics from Google Analytics. The issue may originate from the ETL processor responsible for collecting Internal Searches.

To fix this problem run the following rake task:

rake etl:repopulate_searches["YYYY-MM-DD","YYYY-MM-DD"]

ETL :: no feedex for yesterday

This means the the ETL master process that runs daily has failed to collect feedex metrics from support-api. The issue may originate from the ETL processor responsible for collecting Feedex comments.

To fix this problem run the following rake task:

rake etl:repopulate_feedex["YYYY-MM-DD","YYYY-MM-DD"]

Other troubleshooting tips

For problems in the ETL process, you can check the output in Jenkins.

You can also check for any errors in Sentry or the logs in kibana

This page was last reviewed on 13 January 2020. It needs to be reviewed again on 13 July 2020 by the page owner #govuk-platform-health .
This page was set to be reviewed before 13 July 2020 by the page owner #govuk-platform-health. This might mean the content is out of date.