Table of contents
This page describes what to do in case of an Icinga alert. For more information you could search the govuk-puppet repo for the source of the alert

ntp drift too high

The Network Time Protocol (NTP) daemon (ntpd) is responsible for keeping system time synchronised with standard time servers on the Internet.

The ntpd.drift file records the latest estimate of clock frequency error. If this value gets too high, this can be an indicator of large error between the system clock and the real time.

If the system clock is out of sync on a Vmware host, first make sure the synchronisation between guest and host has been disabled. This is enabled by default in the image configuration and could cause issues when ntpd is running on the same box. To check the status, enable or disable Vmware periodic time synchronization, we can use the vmware-toolbox-cmd program:

$ sudo vmware-toolbox-cmd timesync status
  Enabled
$ sudo vmware-toolbox-cmd timesync disable
  Disabled
$ sudo vmware-toolbox-cmd timesync status
  Disabled

If the Vmware timesync is disabled, we can use fabric-scripts tasks to help resynchronise the system clock:

fab $environment -H jumpbox-1.management ntp.status
fab $environment -H jumpbox-1.management ntp.resync

Note

The fab script will try to slew the time offset, which means continually adding/subtracting little bits of time until the clock is in sync. This is in contrast to a step change, where the clock’s time is just changed. Step changes can cause - for example - log timestamp inconsistencies.

According to the ntpdate man page, the slew forced by the -B flag can take hours to gradually take effect.

Additionally, the ntpdate functionality has been made available in the ntpd program. To resync an offset bigger than 1000, you can run sudo service ntp stop; sudo ntpd -gq; sudo service ntp start.

More about Icinga alerts

This page is owned by #2ndline and needs to be reviewed