Skip to main content
  1. Home
  2. Apps
  3. govuk_publishing_components
Last updated: 19 Oct 2023

govuk_publishing_components: Real User Metrics

Real user metrics (RUM) allow a user's browser to report on how a page loaded using the Performance API. The tool that we use to do this is called LUX - short for Live User Experience - and is run by SpeedCurve.

The benefit of RUM is that it shows how pages are performing in real situations, rather than on synthetic tests. This means that we don't have to guess at what conditions might be - RUM provides insight into what conditions actually are.

Loading

The LUX scripts should only be loaded when a user has opted into usage tracking. This is done using the rum-loader script, which checks if the usage cookie is true and then loads the required scripts. This is already part of the Public Layout component.

How it works

The scripts for the real user metrics are loaded from our servers - this allows us to know what is contained within the scripts and reduce the risk of unwelcome things being added to GOV.UK. Normally LUX is loaded from SpeedCurve's servers, so our setup is unusual.

The scripts

There are three scripts involved.

  • the loader is a script that we wrote. If cookies have been accepted, it finds the LUX script, attaches it to the DOM and executes it. If cookies haven't been accepted, it sets a listener for the cookie-consent event.
  • the measurer measures performance from the start of page load, but doesn't report anything. This file shouldn't ever change. The source is from the RUM dashboard, in Settings, Edit RUM, at the end. We don't use this in the way SpeedCurve recommends, because we load the scripts ourselves.
  • the reporter is the script that the loader pulls down and attaches to the DOM. This file is compiled as a separate asset and isn't included until users consent to cookies.

The reporter is the file that SpeedCurve occasionally changes (in SpeedCurve's repo it's called lux.js). Because we load our own modified copy of this script we currently have to manually download and update it. The script is audited before being updated and then two extra lines are added to make sure it works correctly. These two lines set the customer ID and set the sampling rate.

Customer ID

The customer ID is an identifier for the site using LUX, not for the user visiting the site. It won't change from page to page, or from visitor to visitor.

When loading lux.js from SpeedCurve's CDN, the customer ID is appended to the end of the URI as a query string. The script looks for a script in the DOM with a source of lux.js, and from that extracts the customer ID.

Rails adds a fingerprint to the URI which means that lux.js becomes (for example) lux.self-7137780d5344a93190a2c698cd660619d4197420b9b1ef963b639a825a6aa5ff.js and the script can't find itself. Because of this that part of the script would fail. Instead, we modify the getCustomerId() function to simply return LUX.customerid.

Because of this the customer ID needs to be set in the lux.js file:

LUX.customerid = 47044334

Sending information

LUX needs the content security policy in govuk_app_config to be configured to allow communication to LUX servers. Documentation is provided on how to configure this.

Sample rate

LUX defaults to sending every event to SpeedCurve - this can be changed by setting LUX.samplerate to a integer:

LUX.samplerate = 1 // Whole number from 1 to 100.

This then only sends 1% of visits.

This must be set at the top of the main LUX function or it will default to 100% sample rate.

Debugging (bonus mode)

LUX.debug = true

Debug is turned off by default - setting this to true it will log what LUX is doing in the browser's console.

Usefully, running LUX.getDebug() in the browser's console will show the history of what's happened whether debug is true or false:

LUX.getDebug()

Sampling can also be forced by placing LUX.forceSample() at the end of the file:

LUX.forceSample()

How to update to a new version

Being notified

2nd line have an icinga alert set up to tell us when a new version of LUX is available. This checks the live LUX script for a variable called SCRIPT_VERSION. Since the file is minified it is actually looking for the pattern V=[version_number], where version_number is currently 302. If it does not match this, an alert is triggered.

The icinga alert was originally added in this PR, for reference.

Getting the new code

You will need to download the new script from the SpeedCurve github repo. It's worth turning on notifications for this repo in case the icinga alert fails.

The script is written in TypeScript so you will first need to npm install and npm run build to create a JavaScript version of the file (see repo README for details).

Updating

Update lux-reporter.js by copying dist/lux.js from the SpeedCurve github repo. This is a case of copying and pasting, reformatting, and updating to include the variables mentioned earlier. Try to format the file so that indenting differences don't make reviewing difficult.

Instructions for how to update are in our copy of the file. In summary:

  • customerid and samplerate are the things we need to set (copy them from the current version)
  • customerid has to be inside the LUX = (function () { declaration
  • update the getCustomerId function to simply return the customerid, see this PR for related information

Testing

You can test the changes locally to ensure the file has been updated correctly.

  • temporarily set the debug option in the reporter to true for testing purposes
  • load a local application pointed at your local components gem, and accept cookies
  • paste this command into the browser console: copy(window.LUX.getDebug()) (it'll say undefined but the output will be in your copy/paste buffer)
  • visit the SpeedCurve debugging tool and paste the output into it, and check the output for errors

If you're using docker to run the application, your local server will have a dev.gov.uk domain name and it is possible to set up SpeedCurve to see the data from it. You will need to uncomment the forceSample line to make this work.

Protocol measuring

We've added an extra bit to the end of the measurer script to determine what protocol is in use - HTTP 1, 2 or 3. This shouldn't need any updating as the measurer shouldn't change.