Monit summary is slow/hangs for 1m15s

Question

On the server I’m working with (Ubuntu 14.04 LTS), Monit (v5.6) is hanging for 1 minute 15 seconds before the output of monit summary appears. Other operations, like unmonitor take even longer. I’m at a loss for why, as this is an Amazon EC2 c4.4xlarge instance with no significant CPU/memory load.

Other servers in this cluster don’t have this problem.

Here are some scenarios with time output:

1. With lots of things being monitored:

root@SERVER01:~# time monit summary
...
Process 'crond'                     Running
File 'cron_bin'                     Accessible
File 'cron_rc'                      Accessible
Directory 'cron_spool'              Accessible
System 'SERVER01'                   Running

real    1m15.091s
user    0m0.000s
sys     0m0.011s

2. Disabling monitoring of everything

root@SERVER01:~# time monit unmonitor all

real    6m30.446s
user    0m0.026s
sys     0m0.000s

3. Same slowness with everything unmonitored

root@SERVER01:~# time monit summary
The Monit daemon 5.6 uptime: 36m

Process 'rsyslogd'                  Not monitored
...
Process 'crond'                     Not monitored
File 'cron_bin'                     Not monitored
File 'cron_rc'                      Not monitored
Directory 'cron_spool'              Not monitored
System 'NC-CRIMSON01'               Not monitored

real    1m15.089s
user    0m0.010s
sys     0m0.003s

It's strange that the result always takes 1 minute 15s. Lowering set daemon 90 to 30 had no effect.

Answer 1

Ok, so I was able to resolve this myself, but I'll post it for the benefit of the Internet in general.

After noticing that http://localhost:2812 was taking 17s to respond to each request, it made me consider DNS and timeouts...

It turns out that this new server did not have localhost present in it’s /etc/hosts! ಠ_ಠ