smokey: Tagging
@... tags should only be used for filtering (not functionality hooks).
@app-<app_name>
The Smokey job in Jenkins has an option to filter scenarios by application, so that we can avoid running the entire test suite when only some scenarios are relevant to a change. To associate tests with an app, you need to tag the relevant scenarios (or entire feature) with @app-<app_name>.
Ensure
<app_name>matches the name of the app's Argo CD application resource undergovukApplicationsinvalues-<environment>.yaml. If these don't match then the scenario will not block promotion of a release of that app to staging/production.
@not<environment>
All features should run in all environments. Sometimes this isn't possible, for example where:
-
The actual functionality only exists in
Production.
Tag the feature or scenario with
@notintegrationand@notstagingso it doesn't run in these environments i.e. its only runs in Production. - The test creates or modifies content which can't yet be done in production.
@pending
This will stop a feature or scenario from running in all environments.
Only use this tag if you know you will re-enable it soon. Otherwise, it's clearer to just remove it - either way you will need to make a PR to restore it, so the amount of work is similar.
@local-network
This will stop a feature or scenario running on a local machine. Use this tag if there's a network policy in place that prevents your local machine from accessing protected functionality.
Before adding this tag, check if you could write a better test. It's hard to maintain tests that can't be run locally, so only use this tag if there's no practical way to fix the access issue.