OPNsense Forum

Archive => 20.7 Legacy Series => Topic started by: Taomyn on July 26, 2020, 11:37:08 AM

Title: Monit no longer starts
Post by: Taomyn on July 26, 2020, 11:37:08 AM
Since upgrading to v20.1.9 the Monit service will no longer start:

2020-07-26T11:15:38   monit: /usr/local/etc/monitrc:14: syntax error 'mail-format'
2020-07-26T11:15:30   root: /usr/local/etc/rc.d/monit: WARNING: failed to start monit


Any ideas?

The only place I see "mail-format" looks like this:

(https://forum.opnsense.org/index.php?action=dlattach;topic=18259.0;attach=11306;image)

I also have this constantly at the top of the each settings tab, and apply does nothing:

(https://forum.opnsense.org/index.php?action=dlattach;topic=18259.0;attach=11308;image)
Title: Re: Monit no longer starts
Post by: franco on July 27, 2020, 11:51:55 AM
Is this a monit change? I would prefer trusting binary upgrades *sighs*

# opnsense-revert -r 20.1.8 monit

Does this bring it back to life?


Cheers,
Franco
Title: Re: Monit no longer starts
Post by: Taomyn on July 27, 2020, 12:03:52 PM
Nope, still the same error. When I went check the settings the "Apply" banner had gone but the service would not start. So I forced a save, the banner came back and won't go away again.


Do I need to reboot?
Title: Re: Monit no longer starts
Post by: franco on July 27, 2020, 12:17:12 PM
No, this is strange. What's in /usr/local/etc/monitrc line 14? Maybe a third party library parsing code changed and now it thinks there's invalid input there.
Title: Re: Monit no longer starts
Post by: Taomyn on July 27, 2020, 12:25:54 PM

It's



set alert ferd@mydomain.com not on  mail-format { From: bart@mydomain.com } reminder on 10 cycles


And this is the whole file



# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


set httpd unixsocket /var/run/monit.sock
    allow localhost


set daemon 120 with start delay 120


set logfile syslog facility log_daemon






set mailserver 192.168.1.10 port 25   


set alert ferd@mydomain.com not on  mail-format { From: bart@mydomain.com } reminder on 10 cycles


check system bart.mydomain.com
   if changed status then alert
   if cpu usage is greater than 75% then alert
   if loadavg (1min) is greater than 8 then alert
   if loadavg (5min) is greater than 6 then alert
   if loadavg (15min) is greater than 4 then alert
   if memory usage is greater than 75% then alert
   if failed link then alert
   if space usage is greater than 75% then alert


check filesystem RootFs with path "/"
   if space usage is greater than 75% then alert


check program carp_status_change with path "/usr/local/opnsense/scripts/OPNsense/Monit/carp_status" timeout 300 seconds
   if changed status then alert


check program gateway_alert with path "/usr/local/opnsense/scripts/OPNsense/Monit/gateway_alert" timeout 300 seconds
   if status != 0 then alert
Title: Re: Monit no longer starts
Post by: Taomyn on July 27, 2020, 12:28:55 PM
FYI, if I clear "Mail format", the syntax error moves to "Reminder"
Title: Re: Monit no longer starts
Post by: pouakai on July 28, 2020, 10:31:11 AM
Looking at the Monit Manual, the syntax on line 14 is wrong.

https://mmonit.com/monit/documentation/monit.html#ALERT-MESSAGES (https://mmonit.com/monit/documentation/monit.html#ALERT-MESSAGES)

The 'not on' should be followed by an 'event', and it is not.

On the web GUI, I think you need to uncheck 'not on', or choose an event (or events) from the drop down box.  I am not sure that the help text of 'leave empty for all events' is correct. 
Title: Re: Monit no longer starts
Post by: Taomyn on July 28, 2020, 10:42:13 AM
Currently the line in the config is:


set alert ferd@mydomain.com not on { uptime } mail-format { From: bart@mydomain.com } reminder on 10 cycles



the rest is still the same and the latest error is:


monit: /usr/local/etc/monitrc:17: syntax error 'changed'
Title: Re: Monit no longer starts
Post by: pouakai on July 28, 2020, 10:53:22 AM
O.K., so now Line 14 is working.

But there is now an error on Line 17,  Monit doesn't like the 'change' syntax.  Should there be another term here - what status changed? 

I've learnt that Monit is very picky about syntax and constructs and I have found this a struggle.  I think you need to study the Monit Manual.
Title: Re: Monit no longer starts
Post by: Taomyn on July 28, 2020, 11:05:56 AM

Yes, but why now because all I did was upgrade then it broke, and now downgraded and still broken. I'm using the GUI to enter any changes, but can it be that broken to not be generating correct settings?

Quote from: pouakai on July 28, 2020, 10:53:22 AM
O.K., so now Line 14 is working.

But there is now an error on Line 17,  Monit doesn't like the 'change' syntax.  Should there be another term here - what status changed? 

I've learnt that Monit is very picky about syntax and constructs and I have found this a struggle.  I think you need to study the Monit Manual.
Title: Re: Monit no longer starts
Post by: Taomyn on August 01, 2020, 11:46:13 AM
Still happening after upgrading to 20.7


Can we get this thread moved over to the production forum?
Title: Re: Monit no longer starts
Post by: Taomyn on August 06, 2020, 01:31:02 PM
Thanks for moving the thread.


So does anyone know why I still cannot start Monit?


I'll happily revert it back to original settings if someone can tell me the steps an what files to delete.
Title: Re: Monit no longer starts
Post by: mimugmail on August 06, 2020, 01:52:49 PM
You need to post your config, otherwise not possible to help
Title: Re: Monit no longer starts
Post by: Taomyn on August 06, 2020, 03:18:54 PM
Quote from: mimugmail on August 06, 2020, 01:52:49 PM
You need to post your config, otherwise not possible to help


Did you look at the version posted earlier in this thread?


In the meantime I was able to narrow it down to one of the service settings, if I attempt to enable any of the "Tests" it fails - leave it at "Nothing selected" and it's fine.


This fails:


2020-08-06T15:14:20 monit[76762]: /usr/local/etc/monitrc:17: syntax error 'changed'





# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


set httpd unixsocket /var/run/monit.sock
    allow localhost


set daemon 120 with start delay 120


set logfile syslog facility log_daemon






set mailserver 192.168.1.10 port 25   


set alert ferd@mydomain.com    reminder on 10 cycles


check system bart.mydomain.com
   if changed status then alert


check filesystem RootFs with path "/"
   if space usage is greater than 75% then alert


check program carp_status_change with path "/usr/local/opnsense/scripts/OPNsense/Monit/carp_status" timeout 300 seconds
   if changed status then alert


check program gateway_alert with path "/usr/local/opnsense/scripts/OPNsense/Monit/gateway_alert" timeout 300 seconds
   if status != 0 then alert





This does not





# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


set httpd unixsocket /var/run/monit.sock
    allow localhost


set daemon 120 with start delay 120


set logfile syslog facility log_daemon






set mailserver 192.168.1.10 port 25   


set alert ferd@mydomain.com    reminder on 10 cycles


check system bart.mydomain.com


check filesystem RootFs with path "/"
   if space usage is greater than 75% then alert


check program carp_status_change with path "/usr/local/opnsense/scripts/OPNsense/Monit/carp_status" timeout 300 seconds
   if changed status then alert


check program gateway_alert with path "/usr/local/opnsense/scripts/OPNsense/Monit/gateway_alert" timeout 300 seconds
   if status != 0 then alert


Title: Re: Monit no longer starts
Post by: pouakai on August 08, 2020, 12:37:09 AM
The GUI does not check what you select.  It will let you select options or tests that do not work.  You will only find out when it won't start up.  Again, you need to read the Monit Manual (and look at the examples), for guidance as to what will work.

I think the 'changed' switch is for checking the output status of shell scripts, not the status of the system.

This is what I have under 'check system':

check system $HOST
   if memory usage is greater than 75% then alert
   if cpu usage is greater than 75% then alert
   if loadavg (1min) is greater than 8 then alert
   if loadavg (5min) is greater than 6 then alert


I haven't changed it; this was what Opnsense installed.

Make sure you have the service tests CPUUsage, LoadAvg1, LoadAvg5 and LoadAvg15 set up under Service Test Settings - they are installed by default, so should be there.
Title: Re: Monit no longer starts
Post by: Taomyn on August 08, 2020, 05:47:02 AM
So I don't need the entry I have for my firewall in the service settings section? Service Test Settings should take care of it as they are all there as you state?
Title: Re: Monit no longer starts
Post by: pouakai on August 08, 2020, 07:20:06 AM
You need both.  The actual tests are in Service Test Settings.  The service is set up in Service Settings.

When setting up a 'service', the 'tests' are selected with the drop down box in Service Settings.

Don't modify the 'tests' in Service Test Settings, unless you have read the Monit Manual.

Don't select the 'ChangedStatus' test for the 'Host' service in Service Settings.  As you have found out, it won't work.
Title: Re: Monit no longer starts
Post by: Taomyn on August 08, 2020, 11:32:20 AM
Seems the GUI is majorly broken because everything was fine with this config before 20.1.9


I tried to add some entries, I got as far as CPUUsage and MemoryUsage, but trying to add SpaceUsage brings back the problem.
Title: Re: Monit no longer starts
Post by: tam on December 21, 2020, 07:27:54 PM
I still get this error: /usr/local/etc/monitrc:14: syntax error 'mail-format'

OPNsense 20.7.7_1-amd64

Any solution? Thanks.
Title: Re: Monit no longer starts
Post by: mimugmail on December 22, 2020, 06:05:41 PM
Just look at the file and google the correct syntax , maybe also in man page. Sometimes it's only a missing space
Title: Re: Monit no longer starts
Post by: Fright on December 22, 2020, 06:22:51 PM
QuoteI still get this error: /usr/local/etc/monitrc:14: syntax error 'mail-format'
whats in the "mail format" option of alert settings?
Title: Re: Monit no longer starts
Post by: pouakai on December 22, 2020, 08:52:43 PM
The web GUI will let you enter settings that don't actually match each other - and then Monit won't work.

You definitely cannot adjust the web settings without reading the Monit manual.  I've learned that myself the hard way.  It's also best to look at the file at /usr/local/monitrc and make sure it matches what is shown in the Monit manual.

Here is part of my Monit file (The rest is custom alert scripts of my own making):


set daemon 120 with start delay 120

set logfile syslog facility log_daemon



set mailserver mydomain.com port 25   

set alert user@mydomain.com   mail-format { from: opnsense_box@mydomain.com } reminder on 10 cycles

check system $HOST
   if memory usage is greater than 75% then alert
   if cpu usage is greater than 75% then alert
   if loadavg (1min) is greater than 8 then alert
   if loadavg (5min) is greater than 6 then alert

check filesystem RootFs with path "/"
   if space usage is greater than 75% then alert


Line 14 (your error line) starts with 'set alert... mail format...'
Title: Re: Monit no longer starts
Post by: tam on December 23, 2020, 11:58:58 AM
Quote from: Fright on December 22, 2020, 06:22:51 PM
whats in the "mail format" option of alert settings?

Subject: $SERVICE on $HOST failed

Fixed my error now by changing the event in the alerts settings. Thanks for the hints.