Application: content-data-api

Data warehouse that stores content and content metrics to help content owners measure and improve content on GOV.UK

• GitHub: content-data-api
• Ownership: #govuk-platform-security-reliability-team
• Category: APIs
• Links:
  • Sentry (error tracking)
  • App dashboard: request rates, errors, durations
  • Kibana App Logs (Production, last 3 hours)
  • Kibana Worker Logs (Production, last 3 hours)
  • Argo CD (content-data-api)

README

A data warehouse that stores content and content metrics, and exposes this information via an API, to help content owners measure and improve content on GOV.UK.

Data is combined from multiple sources, including Publishing API, Feedback and Google Analytics.

Live examples

• List all metrics
• Last month of data for all organisations

Nomenclature

• Data warehouse - The database where we store all the metrics
• Fact - A record containing measurements/metrics
• Dimension - A characteristic that provides context for a fact (such as the time it was extracted, or the content item it belongs to)
• Star schema - The way we structure data in the data warehouse using fact and dimension tables
• ETL (extract, transform, load) - How we get data into the data warehouse

Technical documentation

This is a Ruby on Rails app, and should follow our Rails app conventions.

You can use the GOV.UK Docker environment to run the application and its tests with all the necessary dependencies. Follow the usage instructions to get started.

Use GOV.UK Docker to run any commands that follow.

Running the test suite

To run the test suite:

bundle exec rake

Further documentation

• Importing production data
• Running the ETL process locally

Licence

MIT License