So last week we suffered a power outage. My UPC managed to shut down our Windows Server 2012 Host without a problem. However, I forgot that this host is running my Nagios XI VM. That VM did not shut down correctly. Whilst Nagios does work fine once booted up, the screens are plastered with messages about crashed tables (Nagios uses MySQL to store data).
For those who don’t know, Nagios is an excellent resource/server monitoring platform, a very highly configurable one at that. So it’s a pretty essential piece of my monitoring and security workflow. I do have munin & monit on most of my servers so I can get by, but those are on-host. The Nagios install is at my local location and communicates with the servers via SNMP.
Anyway, enough background. It turns out that it is quite easy to solve the issue. Nagios includes a shell script for this very purpose. All that’s needed is to
- log in to your Nagios VM via SSH or directly
- Head into /usr/local/nagiosxi/scripts/
- Run ./repair_databases.sh
The system now examines the tables and repairs as needed.
QED. Happy Monitoring!