GOV.UK Search: how it works
There are two common kinds of public-facing ‘search’ service on GOV.UK:
The main “site search”, available at https://www.gov.uk/search/all:
- It is powered by search-api-v2…
- …except for when no search term has been provided, in which case it falls back to using search-api.
“Finders” (or “Specialist Finders”)
- Powered by search-api.
- Finders can be published by:
- Search API, e.g. https://www.gov.uk/search/research-and-statistics (see configuration)
- Whitehall, e.g. https://www.gov.uk/government/people (see configuration) (legacy - we now encourage non-specialist finders to be published from Search API)
- Specialist Publisher, e.g. https://www.gov.uk/cma-cases (see configuration)
Both are rendered by finder-frontend.
In addition, there is also the Search API endpoint itself, publicly available at https://www.gov.uk/api/search.json, powered by search-api. See Using the search API for examples of how to use this.
Search API versus Search API V2
search-api-v2 was built to improve the quality of search results for the majority of GOV.UK users (when compared with search-api) and retains a “minimally compatible” API with search-api. It uses Google Cloud Platform (GCP)‘s Vertex AI Search (“Discovery Engine”) product as its underlying search engine.
search-api uses an old version of Elasticsearch as its underlying search engine. There is currently no clear roadmap for retiring search-api.
How content gets into search
Content is published via Publishing API. After an edition is changed, Publishing API publishes a message to the published_documents topic exchange it configured on startup. Interested parties, such as search-api and search-api-v2, subscribe to this exchange to perform post-publishing actions.
search-api and search-api-v2 listen to the publishing queue using the govuk_message_queue_consumer gem. In search-api-v2, PublishingApiMessageProcessor processes the indexing of the content, whereas in search-api, the work is done by MessageProcessor.
In addition to GOV.UK content, GDS staff can insert “external links” into search via Search Admin.
Legacy means for getting content into search
search-api (v1) also has some 'legacy’ means of getting content into its indexes. Whitehall makes some calls to Search API directly (typically for ‘non-editioned’ content), via Whitehall::SearchIndex, which is called by any model that includes the Searchable module. (This legacy behaviour is recognised tech debt and should be removed).
Note that there shouldn’t be a situation where Whitehall submits content to Search API both directly and via Publishing API: the Search API’s ‘migrated formats’ file controls which document types Search API expects from each source. There’s a non_indexable section at the bottom that includes all of the Whitehall document types. Search API checks when processing messages from Publishing API whether or not the document type is indexable, and ignores them if it’s not.
Search Admin
The Search Admin application gives GDS staff a means of inserting “external links” into search. For example, searching for “Complain about bus services - Bus Users” surfaces a link to a non-GOV.UK URL.
As of March 2025, Search Admin is undergoing improvements. Watch this space!
Known limitations
- It isn’t possible to see draft content in search results served by Search API v1 on the draft stack
- Non-English content isn’t supported