My simple monit rules not working as expected

[Martin Konečný]

Hello, I'm monitoring an process that occasionally freezes hard and it is not enough for me to check if it is simply running. I need to connect to the process via telnet and verify it is actually responding to requests. I have the following rules:

 check process airtime-liquidsoap
    with pidfile "/var/run/airtime-liquidsoap.pid"
    start program = "/etc/init.d/airtime-liquidsoap start" with timeout 5 seconds
    stop program = "/etc/init.d/airtime-liquidsoap stop"
    if mem > 600 MB for 3 cycles then restart
    if failed host localhost port 1234
      send "version\r\nexit\r\n"
      expect "Liquidsoap"
    with timeout 2 seconds retry 3 for 2 cycles
    then restart 

This morning I found that the process had frozen again and stayed in that state, meaning there is something wrong with my monit config?

My log is repeating non-stop with the following:

[UTC Mar 22 15:00:33] info     : monit daemon at 255 awakened

[UTC Mar 22 15:00:33] info     : Awakened by User defined signal 1

[UTC Mar 22 15:00:33] info     : 'airtime-liquidsoap' monitor action done

[UTC Mar 22 15:00:43] error    : 'airtime-liquidsoap' process is not running

[UTC Mar 22 15:00:43] info     : 'airtime-liquidsoap' trying to restart

[UTC Mar 22 15:00:43] info     : 'airtime-liquidsoap' start: /etc/init.d/airtime-liquidsoap

[UTC Mar 22 15:00:44] info     : 'airtime-liquidsoap' monitor on user request

[UTC Mar 22 15:00:44] info     : monit daemon at 255 awakened

[UTC Mar 22 15:00:48] error    : 'airtime-liquidsoap' failed to start

[UTC Mar 22 15:00:48] info     : Awakened by User defined signal 1

[UTC Mar 22 15:00:48] info     : 'airtime-liquidsoap' monitor action done

[UTC Mar 22 15:00:58] error    : 'airtime-liquidsoap' process is not running

[UTC Mar 22 15:00:58] info     : 'airtime-liquidsoap' trying to restart

[UTC Mar 22 15:00:58] info     : 'airtime-liquidsoap' start: /etc/init.d/airtime-liquidsoap

[UTC Mar 22 15:00:58] info     : 'airtime-liquidsoap' started

[UTC Mar 22 15:00:58] info     : 'airtime-liquidsoap' monitor on user request

[UTC Mar 22 15:00:58] info     : monit daemon at 255 awakened

I had to run

/etc/init.d/airtime-liquidsoap stop

and 

/etc/init.d/airtime-liquidsoap start

manually to get everything back in order. Any ideas?

--

Open source radio in the cloud. Get yours now! ---> http://airtime.pro

Martin Konecny
Software Developer, Sourcefabric
address@hidden

720 Bathurst St. Suite 203
M5S 2R4, Toronto, ON, Canada
+1 (416) 892-8420
Skype: martin.konecny15

http://www.sourcefabric.org
http://www.twitter.com/Sourcefabric