Application: govuk-knowledge-graph-search
GovSearch web app allowing publishing users to search for content in the knowledge graph
- GitHub
- govuk-knowledge-graph-search
- Ownership
- #data-insights-team
- Category
- Data science
- Links
README
Gov Search is a search engine for GOV.UK, with advanced functionality for content designers. This repository includes the code of the GovSearch front-end.
This is an ExpressJS application written in TypeScript. It shows the user a search interface and queries the backend to fetch and display search results.
The full documentation is available in the Data Community Tech Docs.
Running locally
-
Prerequisite: you need to be logged in to gcloud (using the
gcloud
command) and have access to BigQuery (specifically you need theBigQuery Viewer
andBigQuery Job Runner
roles). Ask a BigQuery admin to add you. -
Clone this repository
-
Run
npm install
to install all dependencies -
Install Sass and compile the Sass sources to CSS with
sass ./src/frontend/scss/main.scss > ./public/main.css
-
Install webpack and compile the browser-side Typescript code to JavaScript by just running
webpack
-
Copy assets from the GOVUK Frontend run
npm run copy-assets
-
Set an environment variable called PROJECT_ID to the name of the GCP project your server will be running on. This is so the server knows where to connect to to run searches get the data. For instance, use
govuk-knowledge-graph-dev
to get the data from the development backend. -
Set an environment variable called
ENABLE_AUTH
to"false"
(or anthing but"true"
, or don’t set one at all), as you won’t need authentication on your local machine -
Start the server with
npm run dev
. -
Point your browser to
https://localhost:8080
(the port can be changed using thePORT
environment variable)
Developing
Files
-
src/backend/app.ts
: the main server file. All the.ts
files in the same folder are server-side code. -
src/backend
: the server-side files. -
src/frontend
: the main browser-side files.webpack
compiles everything topublic/main.js
. -
src/scss/main.scss
: the Sass file thatsass
compiles topublic/main.css
-
./public/assets
: publicly served fonts and images
Software architecture
Mostly for historical reasons, much of the functionality offered runs browser-side. That’s why the application is more JavaScript-heavy than your usual alphagov
app. Although the JavaScript code is generated from TypeScript sources, it doesn’t use any framework like React.
The browser-side code uses the Elm Architecture model: the whole application’s state is held in a variable called state
, and a function called view
renders the HTML that corresponds to the current value of state
, and sets event handlers on that HTML. Whenever an event happens (user clicks on a button, or a search returns) the handleEvent
function updates the state accordingly and runs view
again. This forms the main interaction loop. For instance:
- The user enters search terms and clicks “search”
- The DOM event handler (defined in
view.ts
) triggered runshandleEvent
, which:- retrieves the new search terms from the form
- and updates the state (specifically
state.selectedKeywords
) with the new values - starts the search in BigQuery via the API offered by the ExpressJS server.
- and meanwhile calls view to show the “Please wait” message.
- Eventually the API call returns and triggers
handleEvent
, which updatesstate
with the search results -
handleEvent
also callsview
-
view
renders the state, including the search results. - The page waits for the next event etc.
Running tests
unit tests
We use Jest
npm run test
end-to-end tests
We use Cypress, which is installed automatically on installing the dev
npm packages. If Chrome is installed on your system it should be as simple as running npx cypress open
for the interactive version and npx cypress run
for the command-line version.
To run a single test file , use --spec
. For instance:
cypress run --spec cypress/e2e/url.cy.ts
Deployment
Staging Deployment
Staging deployment is triggered automatically whenever a pull request is merged into the main branch.
This is made possible by the deploy-staging GitHub Action.
Steps:
- Create your feature/fix on a new branch.
- Create a PR targeting the main branch.
- Ensure the PR passes all the CI checks and has been approved
- Merge
- The action will deploy the changes. CloudRun revisions that have been deployed automatically start with the “main-” prefix.
Manually: You can also trigger the workflow manually in the actions tab of the repo, by selecting the “deploy-staging” workflow.
Production Deployment
Production deployments have to be triggered manually for security reasons.
- After testing the changes in the staging environment, create a PR of the main branch against the production branch.
- Review and approve the PR.
- Merge the PR.
- Wait for completion of the
create-release-tag
GitHub Action. It will create a new release and a tag in the GitHub repository for every production deployment. - Once the release appears in the repository, manually run the production-deploy workflow. Select the
latest
tag from theproduction
branch. - (Optional) Write a custom description for the release in Github.
Please note:
- Deployments from local machine should be limited to the development environment.
- The deploy-staging GitHub Action will run again during the PR from main to production. This is expected and won’t cause issues.
- There may be a slight delay (usually less than a minute) before the new release appears in the repository after the create-release-tag action has completed.
Deployment Steps
- Go to production site https://govgraphsearch.dev/ and view the source.
- Look for the line beginning
<!-- Google Tag Manager (noscript) --><noscript><iframe src="https://www.googletagmanager.com/ns.html?
and note the values of the URL parametersid
andgtm_auth
. They look likeGTM-XXXXXXX
andaWEg5ABBXXXXXXXXXXXXXXXXX
. - Run the script
deploy-to-gcp.sh
located at the root directory - Enter the value of
id
as in step 2 as the GTM tracking ID. - Enter the value of
gtm_auth
in step 3 as the GTM AUTH. - You may be prompted to authenticate with
gcloud run login
, in which case do so and start again. - Choose the GCP region
europe-west2
- Continue. Check in the web console that a revision was deployed, and try using it at https://govgraphsearch.dev.
## Logging
We use Pino for logging. Pino enables structured logging, human-readable formatting, top-notch performance.
⚠️⚠️ Pino doesn’t behave the same as console.log
. ⚠️⚠️
You can’t pass it infinite arguments. Instead, it’s one string at a time. If you want to log an object alongside your log, then pass the object as the first argument, then the string.
e.g
console.log('this', 'is', 'a', 'test')
// =
log.info('this is a test')
console.log('Look at this object', { ...anObject })
// =
log.info({ ...anObject }, 'Look at this object')
console.log('A few objects', { ...obj1 }, { ...obj2 })
// Separate your logs instead
log.info('A few objects')
log.info({ ...obj1 }, 'Object 1')
log.info({ ...obj2 }, 'Object 2')
// This WONT WORK
// Second object will be swallowed as objects go to the first argument only.
log.info(obj1, obj2)
If you want the request object logged alongside:
function middeware(req, res, next) {
req.log.info('A log')
// logs: {...req} A log
}
To use the logger otherwise:
import log from './utils/logging'
log.info('in msg')
log.debug('debug msg')
log.error('error')
log.error(error, 'error')
// If you want to log an object:
log.info({ a: 123, b: 456 }, 'This is a log')