How logging works on GOV.UK
For information on how to log in and view stacks, please see the GOV.UK Logit documentation.
Kibana is the interface for viewing logs in Elasticsearch. Use the Logit interface to log into Kibana.
There’s some documentation on useful Kibana queries for 2nd line.
Each machine runs Elastic Filebeat, and independently ships logs to the Logit-provided logstash endpoint.
Filebeat tails logs every 10 seconds and can output to a variety of sources. It is fully incorporated into the Elastic ecosystem.
We use the
filebeat::prospector defined type to create
the filebeat configuration on each instance.
Logstream and Logship
We have a defined type in our Puppet code which uses logship to tail logfiles. We only use Logstream to send nginx metrics, via statsd, to Graphite.
In the future this will be replaced.
Fastly sends logs to S3 for the www, assets and bouncer services. These can be queried through Athena.