Table of contents

Pingdom Bouncer canary check

Bouncer has a canary route at which queries all the database tables which Bouncer uses to serve all responses to users and checks that those tables are not empty. The canary should return 200; if it doesn’t then errors will be being served to users - see the table below for more details of the errors in each case.

The application page for Bouncer and Transition documents Bouncer’s stack.

Possible causes of errors on the canary route include:

  • DNS problems for or - check Bouncer’s Nginx logs to see which requests are getting through
  • CDN problems resulting in requests for not being correctly passed to
  • bouncer-* or transition-postgresql-slave-1 machines being unavailable
  • Bouncer’s dependencies being missing or having errors in its configuration
  • memory leaks in Bouncer
  • the app being unable to connect or authorise to the database
  • the database or the required tables being missing or empty
  • the bouncer role not having SELECT privileges on the required tables

The database tables checked by the canary route (in order) and the effect on requests for transitioned sites of errors when querying them:

Database table HTTP status codes for requests for transitioned sites

table name when table inaccessible when table missing data ===================== ================================================================================================ ========================== hosts 500 for all requests 404 for all requests sites 500 for all requests 500 for all requests mappings 500 for most requests 404 for most requests whitelisted_hosts 500 for requests which should redirect to non-*** domains 501 for those requests organisations 500 for most requests which should serve a 404 or 410 page 500 for those requests

There are other tables in the transition_production database but they are only used by Transition and not by Bouncer.

This page is owned by #2ndline and needs to be reviewed