Assets: how they work
There are two types of asset files.
Static assets are stylesheets (CSS), JavaScript (JS) and image files which make GOV.UK look the way it does.
Uploaded assets - also called attachments - are files like PDFs, CSVs and images which are uploaded via the publishing apps and attached to documents.
Static assets
These assets are served from https://www.gov.uk/assets with a path associated
with the application that is serving them. For example files within
https://www.gov.uk/assets/frontend are assets for the
Frontend application. These files
are cached by the GOV.UK content delivery network.
These assets are served by the cache machines, these will proxy requests to
the application based on the path. For example /assets/frontend/
will proxy asset requests to Frontend
Uploaded assets
Asset Manager is an API that is called internally
by GOV.UK publishing applications to manage
their uploads. It serves the uploaded assets on
assets.publishing.service.gov.uk.
How uploaded assets are stored and served
Asset files are stored in an S3 bucket (e.g. govuk-assets-production in
production) and Asset Manager instructs nginx to proxy requests to them.
Proxying was chosen, rather than linking directly to S3, to support the
following features:
- Assets are not served until they have been virus scanned; a placeholder image or page is shown for assets that are not finished scanning.
- Assets can be access-limited so that only authorised users can see them.
- Asset files can be replaced, and a request to the original path redirects to the replacement. Currently only Whitehall and Specialist Publisher support this.
Network flow for live assets
Retrieving an uploaded asset involves requests between asset-manager’s nginx proxy, asset-manager’s Rails app and S3.
- User makes a request for
https://assets.publishing.service.gov.uk/media/6ae90b0e5b14ba0182a272a4/example.pdfwhich hits Fastly - Fastly on a cache miss makes a request to Router for
example.pdf - Router makes a request to asset manager’s nginx for
example.pdf - nginx performs an internal redirect to
location @appwhich proxies a request to asset-manager forexample.pdf - asset-manager on successful lookup returns
X-Accel-Redirect: cloud-storage-proxy/...which contains the presigned S3 URL (e.g.govuk-assets-production.s3.us-east-1.amazonaws.com/example.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256...) - nginx performs an internal redirect to
location ~ /cloud-storage-proxy/(.*)which proxies a request to S3 forhttps://govuk-assets-production.s3.us-east-1.amazonaws.com/example.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256... - The
govuk-assets-productionS3 bucket retrievesexample.pdfand returns it back to nginx - nginx returns the response containing the
example.pdfasset back to Router - Router returns the
example.pdfasset back to Fastly which caches it - Fastly returns the
example.pdfasset back to the user
sequenceDiagram
actor User
participant Fastly
participant Router
participant nginx
participant asset-manager
participant S3
User->>Fastly:
activate Fastly
Fastly->>Router:
activate Router
Router->>nginx:
activate nginx
nginx->>nginx: location @app
nginx->>asset-manager:
asset-manager-->>nginx:
Note over nginx, asset-manager: Response Headers:
X-Accel-Redirect:...
nginx->>nginx: location ~ /cloud-storage-proxy/(.*)
nginx->>S3:
S3-->>nginx:
nginx-->>Router:
deactivate nginx
Router-->>Fastly:
deactivate Router
Fastly-->>User:
deactivate Fastly
The network flow for the draft stack. is identical except the request goes to authenticating proxy then to draft router.
File size limit
There is a hard limit of 500 MB for assets.
Publishers may see timeouts if they attempt to upload very large attachments. In the past, we’ve worked around this by uploading a small file (in Whitehall) and then replacing the file in Asset Manager.