This page describes what to do in case of an Icinga alert. For more information you could search the govuk-puppet repo for the source of the alert

Warning This document has not been updated for a while now. It may be out of date.

Last updated: 5 Apr 2022

High zombie procs

You can check what the zombie processes are by SSHing onto the machine and running the command:

ps -A -o user,pid,ppid,state,cmd | awk 'NR==1 || $4=="Z" { print }'

The PPID field gives you the parent process ID, which you may be able to use to diagnose what caused the problem, or at least which process to restart.

If there are a number of zombie processes on one of the backend machines, restarting Imminence (with sudo service imminence restart) may fix the problem; if that doesn’t help, reboot the machine at a convenient time.