Skip to main content
Last updated: 20 May 2024

Pact Testing

Pact is a tool we use for contract testing. Contract testing involves creating a set of tests that are shared between an API (the “provider”) and its users (“consumers”). Contract tests allow consumers to define a contract that it expects its provider to uphold. This means that a consumer can specify what response (both status code and body) they expect from a provider given a certain http call. This is useful to ensure that changes to the provider do not break the interaction between it and its consumers.

For example, a consumer might expect a certain field to be returned in a json response from a provider, and use that field in its application code. Without pact tests, we might be able to remove that field from the provider’s response without knowing that it is a breaking change. Good pact tests will prevent such breaking changes from being made.

Since in tests we do not want to spin up both apps to test the interactions, pact provides us with a “broker” which deals with the interactions between the two apps in test.

For example, the Places Manager API has a “pact” or “contract” with GDS API Adapters:

GDS API Adapters is really a proxy for real “consumer” apps, like Whitehall. The gem includes a set of shared stubs for use in each app’s own tests (example). Using the stubs ensures each app stays in sync with GDS API Adapters, which can then do contract testing on their behalf.

Running Pact tests locally

For a consumer (GDS API Adapters)

Pact tests are run as part of the regular test suite:

bundle exec rake test

Or they can be run on their own:

bundle exec rake pact_test

For a provider

Example for Frontend (provider of /bank-holidays.json):

bundle exec rake pact:verify

You can also test against a specific branch of GDS API Adapters:

env PACT_CONSUMER_VERSION=branch-<branch-of-gds-api-adapters> bundle exec rake pact:verify

Note: when a build runs for GDS API Adapters, the generated JSON pact is pushed to the broker as branch-<branch-name> using the pact:publish:branch rake task; this is why PACT_CONSUMER_VERSION needs to start with branch-.

Alternatively, you can test against local changes to GDS API Adapters:

env PACT_URI="../gds-api-adapters/spec/pacts/gds_api_adapters-bank_holidays_api.json" bundle exec rake pact:verify

Adding tests for a new app

If the app already had some Pact tests, follow the steps for changing existing Pact tests.

  1. Write the consumer and provider tests.

  2. Check the tests pass locally for both provider and consumer

    • CI won’t be able to test them yet as they won’t be pushed to the pact broker.
  3. Merge the consumer tests for the new app.

  4. Merge a GitHub Action (example) into the provider app to verify the pact

    • This will verify the provider app fulfills the contract published by the consumer.
  5. Update the consumer to utilise the provider GitHub action as part of the build process (example)

    • This will verify the provider contract when the consumer builds.

Changing existing Pact tests

Follow these steps in order to change the provider and consumer in tandem.

  1. Make the change to the API (your provider, e.g. Places Manager), in a branch.

    • Its build should fail at the Pact test stage, because it is testing against the default branch of the consumer.
  2. Make the change to the pactfile in the consumer (see ‘Where to define the consumer tests’ below) in a branch.

    • Its build should fail at the Pact test stage, because it is testing against the default branch of the provider.
  3. Confirm locally that the provider can verify against the published consumer pact

    • env PACT_CONSUMER_VERSION=branch-<branch-of-gds-api-adapters> bundle exec rake pact:verify
  4. Merge the provider change

    • We’ve configured failing Pact tests to not block merging, so you should still be able to merge.
  5. Re-run the consumer tests

    • These should pass now that the provider has been updated.
  6. Merge the consumer change

Where to define the consumer tests

As is noted above, GDS API Adapters is often used as the consumer for pact tests. This is because many api calls made within our system are made via GDS API Adapters. Therefore treating GDS API Adapters as the consumer saves us from needing to write pact tests for the same api call in multiple different applications.

When an application calls an API directly, not via GDS API Adapters, this application will be the consumer. For example, Publishing API is the consumer in the pact tests between it and Content Store, since it makes direct calls to Content store.