Architecture of data.gov.uk
The data.gov.uk (DGU) platform is used to publish and view datasets. A dataset contains the metadata for a collection of links to data hosted somewhere on the internet.
Architectural overview of data.gov.uk
The original for this diagram is available on the Platform Health Google Drive and can be edited with draw.io.
data.gov.uk Services
Services owned by data.gov.uk
- CKAN is the publishing app for datasets (‘packages’).
- Find is the public frontend for searching datasets using Opencsearch.
- Publish is a prototype publishing app for datasets. Whilst not public facing, it currently syncs data from CKAN into Opensearch for use in Find.
- Reference is a legacy service that attempts to provide a nomenclature of time intervals, hosted on Heroku.
Services with data.gov.uk sub-domains, but owned by other departments
- Statistics is owned by the Office for National Statistics and was established as part of the Open Data Policy.
- Environment is owned by DEFRA and was created with the Location service as part of the Open Data Policy.
- Land Registry is owned by the same and was created to publish linked data as part of the Open Data Policy.
- Catalog Service for the Web is owned by Ordnance Survey and serves INSPIRE datasets to the EU Geoportal.
- Location Metadata Editor is owned by DEFRA and publishes INSPIRE metadata, which is used to enrich datasets.
- Location came before Location Metadata Editor and was established as part of the UK Location Programme.
- Guidance is a set of manual pages hosted in GitHub, which ought to be migrated into normal GOV.UK docs.
- Contract Finder is now provided by Crown Commercial Service, which ought to have pre-2015 stuff merged in.
- Business is a legacy redirect to Companies House.
Several datasets link to environment.data.gov.uk and require user login to access. Although branded as data.gov.uk, this is a totally separate service. If a user is having difficulty accessing this system, they should contact the maintainers of this resource, who are currently Airbus Defence & Space.
Publish and Find
Publish and Find are provisioned on GOV.UK Paas. The deployment and monitoring pages explain this in more detail, but you can use the following commands to get an overview.
cf apps
cf services
cf routes
cf env publish-data-beta-production
We use GOV.UK Signon for user authentication in Publish Data, with the app in each environment linked to the corresponding instance of GOV.UK Signon. See the Publish ADR for more info.
CKAN
CKAN is hosted on AWS and is maintained/deployed in the same way as most other GOV.UK applications.