Application: search-api
Search API for GOV.UK
- Ownership
- #govuk-platform-health
- Hosting
- AWS, on the search machine class
- Continuously deployed?
- No
Relevant manual pages
- Content that doesn’t show up correctly in search or list pages
- Make a new document type available to search
- Reindex an Elasticsearch index
Imported documents
- Adding new fields to a document type
- Content API
- Content in the search index and where it comes from
- Documents API (to be deprecated)
- How Search Works
- Indexing
- Moving a format to the new indexing process
- Popularity information
- Publishing finders
- Relevancy
- Schema definitions
- Search Quality Metrics
- Using the search API
- learning-to-rank
- /arch
- Decision record: Batch processing of rabbit MQ messages
- Decision record: Incremental popularity updates
- Decision record: Learning To Rank
- Decision record: Sagemaker
- Decision record: Transition mainstream formats to a Publishing API derived search index
- Decision record: Transition whitehall documents to a Publishing API derived search index
- Decision record: Upgrade to Elasticsearch 5
- Decision record: Upgrade to Elasticsearch 6
- Decision record: perform popularity updating without using an index lock
- Decision record: use of both RabbitMQ (bunny) and Redis (Sidekiq) queuing systems
- Decision record: use versioning to handle race conditions when updating the govuk index
- /public-api
Example pages published by search-api
- Search
- News and communications
- Services
- Policy papers and consultations
- Research and statistics
- Guidance and regulation
- Transparency and freedom of information releases
- Official documents
README
Search API (née “rummager”) indexes content into Elasticsearch and serves the GOV.UK Search API.
GOV.UK applications use the API to search and filter GOV.UK content. For example, alphagov/finder-frontend uses the search API to render site search and finder pages (such as gov.uk/aaib-reports).
Search API also provides a public API: https://www.gov.uk/api/search.json?q=taxes.
API documentation
If you would like to use the Search API, please see the Search API documentation.
You can also find some examples in the blog post: “Use the search API to get useful information about GOV.UK content”.
Technical documentation
Search API is a Sinatra application that interfaces with Elasticsearch.
You can use the GOV.UK Docker environment to run the application and its tests with all the necessary dependencies. Follow the usage instructions to get started.
Use GOV.UK Docker to run any commands that follow.
Running the test suite
bundle exec rake
Additional Docs
- How documents are indexed
- How documents are retrieved
- Search relevancy - uses AWS Sagemaker
- New indexing process: how to update a format to use the new indexing process
- Schemas: how to work with schemas and the document types
- Popularity information: Search API uses Google Analytics data to improve search results.
- Publishing document finders: Information about publishing finders using rake tasks
- Learning to rank: Guidance on how to run the ranking model locally