[SOLVED] zabbix/vnstat not starting

[gpb]

Installed as new on my test machine (HP T620 thin client) using hybrid zfs option (and tmp/var in RAM option after restoring config).  Couple issues.

1) Vnstat does not start, has the following error message in backend logs:

[a4036e6c-0f4d-4220-9282-436aa8ed4717] Script action failed with Command '/usr/local/bin/vnstat -m ' returned non-zero exit status 1. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 478, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/bin/vnstat -m ' returned non-zero exit status 1.

Tried changing the interface settings to WAN and then LAN but no difference.  This is running behind my production router...I doubt that should make a difference.

2) Zabbix client won't start.  Noticed it's in plugins twice, once at the top in red indicating the package is missing...

os-zabbix-agent (missing)   1.10   50.1KiB   OPNsense   Zabbix monitoring agent

It's also listed further down as such:

os-zabbix54-agent (installed)   1.10   50.1KiB   OPNsense   Zabbix monitoring agent

Running the automatic resolver in firmware settings, I get this each time.

Code Select Expand

***GOT REQUEST TO SYNC***
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking integrity... done (2 conflicting)
  - os-zabbix-agent-1.10 conflicts with os-zabbix54-agent-1.10 on /usr/local/etc/inc/plugins.inc.d/zabbixagent.inc
  - zabbix5-agent-5.0.19 conflicts with zabbix54-agent-5.4.9 on /usr/local/bin/zabbix_get
Checking integrity... done (0 conflicting)
The following 5 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
os-zabbix54-agent: 1.10
zabbix54-agent: 5.4.9

New packages to be INSTALLED:
os-zabbix-agent: 1.10
zabbix5-agent: 5.0.19

Installed packages to be REINSTALLED:
pkg-1.16.3

Number of packages to be removed: 2
Number of packages to be installed: 2
Number of packages to be reinstalled: 1
[1/5] Deinstalling zabbix54-agent-5.4.9...
[1/5] Deleting files for zabbix54-agent-5.4.9: .......... done
==> You should manually remove the "zabbix" user.
==> You should manually remove the "zabbix" group
[2/5] Deinstalling os-zabbix54-agent-1.10...
[2/5] Deleting files for os-zabbix54-agent-1.10: .......... done
Reloading plugin configuration
[3/5] Installing zabbix5-agent-5.0.19...
===> Creating groups.
Using existing group 'zabbix'.
===> Creating users
Using existing user 'zabbix'.
[3/5] Extracting zabbix5-agent-5.0.19: ......... done
[4/5] Reinstalling pkg-1.16.3...
[4/5] Extracting pkg-1.16.3: .......... done
[5/5] Installing os-zabbix-agent-1.10...
[5/5] Extracting os-zabbix-agent-1.10: .......... done
Stopping configd...done
Starting configd.
Reloading plugin configuration
Configuring system logging...done.
Reloading template OPNsense/ZabbixAgent: OK
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking integrity... done (2 conflicting)
  - os-zabbix54-agent-1.10 conflicts with os-zabbix-agent-1.10 on /usr/local/etc/inc/plugins.inc.d/zabbixagent.inc
  - zabbix54-agent-5.4.9 conflicts with zabbix5-agent-5.0.19 on /usr/local/bin/zabbix_get
Checking integrity... done (0 conflicting)
The following 5 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
os-zabbix-agent: 1.10
zabbix5-agent: 5.0.19

New packages to be INSTALLED:
os-zabbix54-agent: 1.10
zabbix54-agent: 5.4.9

Installed packages to be REINSTALLED:
pkg-1.16.3

Number of packages to be removed: 2
Number of packages to be installed: 2
Number of packages to be reinstalled: 1
[1/5] Deinstalling zabbix5-agent-5.0.19...
[1/5] Deleting files for zabbix5-agent-5.0.19: .......... done
==> You should manually remove the "zabbix" user.
==> You should manually remove the "zabbix" group
[2/5] Deinstalling os-zabbix-agent-1.10...
[2/5] Deleting files for os-zabbix-agent-1.10: .......... done
Reloading plugin configuration
[3/5] Installing zabbix54-agent-5.4.9...
===> Creating groups.
Using existing group 'zabbix'.
===> Creating users
Using existing user 'zabbix'.
[3/5] Extracting zabbix54-agent-5.4.9: ......... done
[4/5] Reinstalling pkg-1.16.3...
[4/5] Extracting pkg-1.16.3: .......... done
[5/5] Installing os-zabbix54-agent-1.10...
[5/5] Extracting os-zabbix54-agent-1.10: .......... done
Stopping configd...done
Starting configd.
Reloading plugin configuration
Configuring system logging...done.
Reloading template OPNsense/ZabbixAgent: OK
Checking integrity... done (0 conflicting)
Nothing to do.
***DONE***

For zabbix, I don't see any messages in the system log.

Thanks!  Great work and exciting.

PS - One question, was sort of expecting DNSCrypt-proxy 2.1.1 which has some new features, it had been requested a while back from at least one other user but it's absent.  I've been manually running the new version in production for past couple months (replacing binary and pointing to a custom config).  I can continue doing this, but just curious.  Thx!

UPDATE: Zabbix was a config setting...it starts now.

[franco]

there's os-zabbix-agent and os-zabbix54-agent. You had the former installed but then installed the latter. The conflict persists since both are not the same. But it's easy enough to clean this up by resetting the plugin state on the firmware status tab (there's an extra drop down right next to the audit button to do this).

Dnscrypt-proxy 2.1.1 has been parked for various reasons. A number of people work on it, but since the plugin and actual software have a lot of proposed changes we don't have a quick way forward.

Not sure about vnstat, maybe it needs a database reset?


Cheers,
Franco

[gpb]

Thanks Franco,

I was able to resolve the zabbix conflict, didn't know about that and wasn't sure how two ended up in there.

Regarding vnstat, this looks like an issue.  This looks related to the use of ramdisk for /var and /tmp and the lack of the symbolic link added to the database folder.  In 21.7.7 I have this:

Code Select Expand

root@opnsense:~ # ls -l /var/lib/vnstat
lrwxr-xr-x  1 vnstat  vnstat  20 Dec 17 17:23 /var/lib/vnstat -> /root/var/lib/vnstat

So that link is missing when installing 22.1b from scratch and importing config (also removing the vnstat plugin and installing clean doesn't add the link).  The database itself does exist in the /root/var/lib/vnstat folder.

I tried manually adding the link with "ln -s /var/lib/vnstat/ /root/var/lib/vnstat/" but it ended up creating a recursive link to itself...?

Anyhow, that appears to be the problem.  How would this be resolved manually?  Thanks!

[franco]

I think this can be reproduced with the information you've given. I had to rework the /var link stuff for mount points that interfere like is the case on ZFS. Let me take a look. :)

The whole /var/lib/vnstat business is a bit fishy though, at one point the plugin might try to "rm -rf /var/lib/vnstat" which is bad news when /var/lib/vnstat is a link to begin with.

In 2022, I can't much recommend /var MFS option anymore unfortunately. It might go away in the not so distant future.


Cheers,
Franco

[gpb]

OK, I was testing this more to help rather than get it working for me.  :)  I can turn off the var ramdisk, that's not a problem, but that option has been working great for the past months.  I am curious though how to manually enter that symlink to "/root/.....".

One other thing that I noticed that I can't figure out and could be related to me being double nat'd (22.1b behind 21.7.7 production)...URL Table aliases in the firewall from spamhaus aren't populating.  No messages on that one.  If no one else reports it, chalk it up as a local issue to my setup (the URL is not being blocked).  Can that be manually triggered for testing?  Or where to check...nothing in system log.  Thx!

[franco]

We try to fix /var MFS issues, but they do pop up quite frequently with newly added plugins for example.

After some head-banging it looks as if /root directory on FreeBSD 13 has changed permission to 750 away from 755, which means vnstat user cannot get to /root/var anymore. It's easily fixed, but not very nice and overwritten on next base update:

# chmod 755 /root

We will need to discuss what to do internally...


Cheers,
Franco

[franco]

Ok, found it https://github.com/opnsense/src/commit/a35bc248fd6

[franco]

One other thing that I noticed that I can't figure out and could be related to me being double nat'd (22.1b behind 21.7.7 production)...URL Table aliases in the firewall from spamhaus aren't populating.  No messages on that one.  If no one else reports it, chalk it up as a local issue to my setup (the URL is not being blocked).  Can that be manually triggered for testing?  Or where to check...nothing in system log.  Thx!

Not an expert on this subject depending on what interval that alias has been set to resolve it could take a while to resolve (check settings first).


Cheers,
Franco

[gpb]

Ahhh...ok.  Will change my /var setting back to "standard" to eliminate these gotchas.  Thanks again.

EDIT: Aliases did finally update.

[franco]

Hi gpb,

We decided to revert the FreeBSD 750 permission change for /root for the time being so with 22.1 this should be back to normal.


Cheers,
Franco

[gpb]

Great, thanks for the update!

Migrated my prod box, and aside from the firefox cache (?) issue trying to get to 22.1R1 I posted about separately, haven't run into any issues.  :)

[athurdent]

Great, thanks for the update!

Migrated my prod box, and aside from the firefox cache (?) issue trying to get to 22.1R1 I posted about separately, haven't run into any issues.  :)

Cache issue with macOS Safari here, too. Had to clear cache multiple times to get the update window to output something/upgrade popups to appear.