Last updated: 12 Apr 2024

specialist-publisher: Creating, editing and removing specialist document types and finders

To create or edit a new specialist document you will have to make changes to this application, publishing-api and search-api. You will not have to make any changes to frontend applications.

Creating a specialist document type

1. Add a schema to Publishing API

See example PR here

1. Add the format to this list and to this list

2. Add any new field definitions to this file

3. Add examples as instructed. You can copy and paste from another specialist document format, only changing what is necessary (you can leave the body and headers unchanged).

   You'll need to generate your own UUIDs for the content_id and signup_content_id fields:

   $ irb
   irb(main):001:0> require "securerandom"
   => true
   irb(main):002:0> SecureRandom.uuid
   => "5087e8b6-ee54-40f9-b592-8c2813c7037d"
   

4. Run bundle exec rake build_schemas to regenerate schemas.

When the PR is reviewed and its tests passing, it can be merged and deployed at this point.

2. Create a new specialist document format in Specialist Publisher

Create the schema

See CMA cases.

New formats are often requested to be deployed in "pre-production mode", which is configured in this step (example). pre-production documents are only publishable on development and integration.

Create the model

See CMA cases

Create the view template

See CMA cases

3. Configure Search API

Search API needs copies of the schema very similar to the one in Specialist Publisher. See:

• CMA case schema (example)
• field definitions

You'll also need to add your document format to:

• the main ES types list govuk.json
• migrated_formats.yaml
• mapped_document_types.yaml

Finally, you'll need to add your custom fields to:

• elasticsearch_presenter.rb
• specialist_presenter.rb

4. Configure the email sign up page

The email sign up page is rendered by Finder Frontend using the configuration in the new schema added to specialist publisher. The schema should specify email_filter_by and email_filter_facets (e.g. cma-cases).

If your email sign up page should have checkboxes (e.g. cma-cases), you will need to edit email-alert-api by adding the new tags to valid_tags.rb.

5. Deploy and publish

To deploy:

1. Deploy Publishing API (if you haven't already), Specialist Publisher and Search API.
   • Ensure you deploy Publishing API first, to avoid schema validation errors.
   • Also deploy Email Alert API if you have made changes to it.
2. Reindex the govuk Elasticsearch index.
   • This takes around 30-45 minutes on Production, or 3-4 hours on Integration.
   • Alternatively, run search:update_schema for a shorter run. Make sure the this is run before any documents are published, otherwise a full reindex will be required.
   • NB: reindexing shouldn't really be necessary; Elasticsearch will dynamically create the field mappings the first time a new document of this type is published. In other words, if you publish a new document type, the finder will work and it will return the relevant documents even without a reindex. However, the filters on the finder would not work, as this reindexing job also builds the filters for the finder, so we have to run the job.
3. Publish the finder; run the rake task publishing_api:publish_finders or publishing_api:publish_finder[your_format_name_based_on_the_schema_file] against the specialist publisher app (rake tasks here).

6. Permissions

Specialist Publisher grants access to the publishing interface for your new document type to the following Signon users:

• Users that belong to the owner organisation AND have Editor permissions
• Users that have the permission your_new_document_type_editor, e.g. oim_project_editor

You'll need to create the new permission manually.

Editing a specialist document type

We often receive requests to add new fields to a specialist document or to add new values to existing fields.

Adding a new field to an existing specialist document

1. In publishing-api:

   • Add the new field in the specialist_document schema, and add the new values. See example commit.
   • Update the facets in the corresponding example json with the additional values.

   NOTE: You will need to run bundle exec rake build_schemas to regenerate schemas after adding the new value(s).

2. In specialist publisher:

   • Add the new field to the relevant model.
   • Add field to relevant view.
   • Add fields to relevant schema files.

   See this commit for an example.

3. In search-api, add the new field in the following places (see this commit for an example):

   • the relevant schema in the elasticsearch_types directory.
   • the elasticsearch_presenter.
   • the specialist_presenter.
   • the field_definitions file.

4. Follow steps in the Deploy and publish section above, to re-publish the finder. It should suffice to run search:update_schema in search-api.

Adding or amending values for existing fields on a specialist document

1. In publishing-api:

   • Add the new values to the field you are amending, in the specialist_document schema. See example here.
   • Update the facets in example json with the additional values, as in this commit.

   NOTE: You will need to run bundle exec rake build_schemas to regenerate schemas after adding the new value(s).

2. In specialist publisher, add the new values to the relevant file in the schema directory. See this commit for an example.

3. In search-api, amend the value in the relevant schema in the elasticsearch_types directory. See this commit for an example.

4. Republish the finder, see step 3 in the Deploy and publish section above. It should suffice to run search:update_schema in search-api.

Editing a specialist finder

The schema files that define a specialist document, are also used to configure that document's specialist finder. See this commit for an example.

Unpublishing a specialist finder

Important: In order to remove a finder, you should ensure there is a clear business requirement in doing so, and that all associated documents have been correctly dealt with - unpublished, redirected, migrated or removed. If the code is not removed from all relevant repositories, there is a chance the finder will be inadvertently republished, for example by republishing all finders. You must therefore make sure the code is removed in order to keep the finder unpublished.

The following steps are required to remove a finder:

1. Unpublish the finder in all environments using the provided rake task:

   rake unpublish:redirect_finder["uk_market_conformity_assessment_bodies","https://redirection_link.gov.uk"]
   

2. Remove usages from publishing-api:

   • Remove the format from this list and this list.
   • Remove the field definitions from this file.
   • Remove the example from this directory.
   • Run bundle exec rake build_schemas to regenerate schemas.

   See example commit.

3. Remove usages from specialist publisher. See example commit.

4. Remove usages from search-api. See example commit.

5. Remove any usages from finder-frontend, if applicable.

6. Deploy all changes.