Manage assets
Removing an asset
If you need to remove an asset manually from assets.publishing.service.gov.uk
,
follow these steps:
Ideally, the deletion should be done in the publishing application that originally uploaded the file (e.g. Whitehall). Deleting an asset from Asset Manager without removing the record from the publishing application could cause problems if the asset is attempted to be re-published again in the future.
If it isn’t possible or desirable to remove the asset in the publishing app, use these steps to remove the asset in Asset Manager:
- Get the asset ID. This can be obtained from the URL for most assets, e.g. the asset ID
for
https://assets.publishing.service.gov.uk/media/65f2c1110e1c2f8c4dffaa53/my_file.jpg
is65f2c1110e1c2f8c4dffaa53
.
For Whitehall assets that do not start /media
, the ID can be obtained using the slug,
e.g. for https://assets.publishing.service.gov.uk/government/uploads/uploaded/hmrc/realtimepayetools-update-v23.xml"
,
run the following in an Asset Manager rails console:
WhitehallAsset.find_by(legacy_url_path: "/government/uploads/uploaded/hmrc/realtimepayetools-update-v23.xml").id
- Decide if the asset can be marked as deleted or whether all traces need to be removed (e.g. if it contains secret information).
To soft delete the asset (i.e. mark the asset as deleted in Asset Manager’s database, but retain the file in cloud storage), run:
k exec deploy/asset-manager -- rake assets:delete[ASSET_ID]
To delete all traces of the asset (including the file from cloud storage), run:
k exec deploy/asset-manager -- rake assets:delete[ASSET_ID, true]
Add a query string to the URL (e.g.
?cache-bust=12345
) to bypass the cache and check that the asset responds with a 404 not found.Wait 5 minutes for the cache to clear, or purge it yourself.
Verify that the URL returns a 404 response without using a query string.
Request removal of the asset from Google’s search results using the Google Search Console.
Remove the asset from the Google Cloud Platform (GCP) mirror:
- Log into the GCP console.
- Go to the
GOVUK Production
project under theDIGITAL.CABINET-OFFICE.GOV.UK
organisation. - Select
Cloud Storage -> Browser
, go to thegovuk-production-mirror
bucket. - Navigate to the file, then delete it.
Remove the asset from the Amazon Web Services (AWS) mirror:
gds aws govuk-production-poweruser aws s3 rm s3://govuk-production-mirror/assets.publishing.service.gov.uk/<slug>
Redirecting an asset
Sometimes it might be necessary to manually redirect an asset, for example if an associated document wasn’t unpublished correctly.
Ideally, the redirect should be done in the publishing application that originally uploaded the file (e.g. Whitehall).
If it isn’t possible or desirable to redirect the asset in the publishing app, use these steps to remove the asset in Asset Manager:
- Get the asset ID. This can be obtained from the URL, e.g. the asset ID for
https://assets.publishing.service.gov.uk/media/65f2c1110e1c2f8c4dffaa53/my_file.jpg
is65f2c1110e1c2f8c4dffaa53
.
For Whitehall assets that do not start /media
, the ID can be obtained using the slug,
e.g. for https://assets.publishing.service.gov.uk/government/uploads/uploaded/hmrc/realtimepayetools-update-v23.xml"
,
run the following in an Asset Manager rails console:
WhitehallAsset.find_by(legacy_url_path: "/government/uploads/uploaded/hmrc/realtimepayetools-update-v23.xml").id
- Run the following rake task:
k exec deploy/asset-manager -- rake assets:redirect[ASSET_ID,REDIRECT_URL]
Also see getting an asset’s ID if you have only been provided the URL.
Uploading an asset
Some publishing apps such as Mainstream Publisher do not provide the facility for editors to upload assets such as images and PDFs. In these rare cases, we can upload assets to asset-manager manually and give the URL to content editors to embed.
Production assets are replicated to staging and integration nightly, so it is best to simply perform the upload directly in production.
If the pod terminates at any point during this process (e.g. a deployment occurs), you will need to start again as the
/tmp
directory is specific to the pod.
- Get the name of the asset-manager pod (you need the app pod, which will be named
like
asset-manager-abc1234-abc123
and does not contain any extra words, such asfreshclam
,pss
orworker
):
$ kubectl -n apps get pods | grep asset-manager
- Assign the pod name to a variable:
$ POD_NAME={name-of-app-pod-from-previous-output}
- Upload the file to the pod’s
/tmp
directory:
$ kubectl cp my_file.jpg apps/${POD_NAME}:/tmp
- Run the upload command:
$ kubectl -n apps exec -it ${POD_NAME} -- bin/create_asset /tmp/my_file.jpg
The output will look like the following:
Saved!
Asset id: 65f2c1110e1c2f8c4dffaa53
Asset name: my_file.jpg
Asset basepath: /media/65f2c1110e1c2f8c4dffaa53/my_file.jpg
The asset can now be located at https://assets.publishing.service.gov.uk/{base-path-from-previous-output}
,
e.g. https://assets.publishing.service.gov.uk/media/65f2c1110e1c2f8c4dffaa53/my_file.jpg
.
Replacing an asset
If you need to replace the file of an existing attachment, follow these steps.
The replacement file must have a different filename to the one it is replacing.
Ideally, the replacement should be done in the publishing application that originally uploaded the file (e.g. Whitehall).
If it isn’t possible or desirable to replace the asset in the publishing app, use these steps to remove the asset in Asset Manager:
- Get the name of the asset-manager pod (you need the app pod, which will be named like
asset-manager-abc1234-abc123
and does not contain any extra words, such asfreshclam
,pss
orworker
):
$ kubectl -n apps get pods | grep asset-manager
- Assign the pod name to a variable:
$ POD_NAME={name-of-app-pod-from-previous-output}
- Upload the file to the pod’s
/tmp
directory:
$ kubectl cp my_file.jpg apps/${POD_NAME}:/tmp
- Get an app console on that same server:
$ kubectl -n apps exec -it ${POD_NAME} -- rails c
- Find the asset:
asset = Asset.find("asset-id-from-url") # e.g. `57a9c52b40f0b608a700000a`
Check the asset is what you think it is.
Replace the file:
asset.file = Pathname.new("/tmp/filename.ext").open
asset.send(:store_metadata)
asset.save!
- If the attachment is a PDF that was originally uploaded in Whitehall, update the page count field in Whitehall:
$ kubectl -n apps exec -it deploy/whitehall-admin -- rails c
attachment_data = Asset.find_by(asset_manager_id: asset-id-from-url).assetable # e.g. `57a9c52b40f0b608a700000a`
attachment_data.update(file_size: 123, number_of_pages: 28) # file size in bytes
Getting an asset’s ID
You can get an asset’s ID by using it’s URL:
Open a Rails console:
k exec -it deploy/asset-manager -- rails c
Find the asset:
asset = Asset.find_by(legacy_url_path: "/slug") id = asset.id
For example, if the URL is
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1234/document.pdf
the slug would be
/government/uploads/system/uploads/attachment_data/file/1234/document.pdf