OPNsense NUT issues

[AbsolutelyFree]

Hello all,

EDIT: See second post, the issue described in my first post was just a symptom of a much bigger issue.

I have stumbled upon a very odd issue when using OPNsense as a NUT server and trying to connect a vanilla FreeBSD client via nut_upsmon. I have provided 4 attachments that show my NUT configuration from the OPNsense UI. I am using a cyberpower brand UPS using the usbhid-ups driver. The passwords shown in my screenshots are sanitized versions of what they actually are. This all seems to work correctly, I can see all of the information from the UPS on the NUT diagnostics page (also shown in the attached screenshots).

I have setup a port forward on OPNsense that is forwarding NUT traffic from my FreeBSD client server to my OPNsense router correctly. I can tell this works because if I run "upsc cyberpower@OPNSENSE_IP" from my FreeBSD server where OPNSENSE_IP is the IP address of my router in the subnet that the FreeBSD server is in, I get output that is the same as what shows in the NUT diagnostics page in OPNsense. I can also see the traffic being passed correctly in the firewall logs in OPNsense.

I want to run the nut_upsmon service on this FreeBSD server to monitor the UPS attached to OPNsense. In my upsmon.conf file, I have:

Code Select Expand

MONITOR cyberpower@OPNSENSE_IP:3493 1 monuser password2 slave

Which I believe is the only relevant configuration when it comes to connecting upsmon to a upsd server. Now here is where things get odd. I run upsmon -D on my FreeBSD host to keep the program in the foreground in debugging mode. This is what I see:

Code Select Expand

Network UPS Tools upsmon 2.8.0
kill: No such process
   0.000000 UPS: cyberpower@OPNSENSE_IP:3493 (secondary) (power value 1)
   0.000200 Using power down flag file /etc/killpower
   0.000476 [D1] debug level is '1'
   0.000484 [D1] debug level is '1'
   0.001023 [D1] Saving PID 93255 into /var/db/nut/upsmon.pid
   0.003924 [D1] Trying to connect to UPS [cyberpower@OPNSENSE_IP:3493]
   0.005561 Login on UPS [cyberpower@OPNSENSE_IP:3493] failed - got [ERR ACCESS-DENIED]


I have spent the past 2 days investigating this issue, but between the fact that everything appears correct in the OPNsense UI, my syntax is correct in upsmon.conf (https://www.freebsd.org/cgi/man.cgi?query=upsmon.conf), that I see the traffic being passed correctly in OPNsense's logs, and that I get results when I run "upsc cyberpower@OPNSENSE_IP" from the FreeBSD server, I have completely run out of ideas.

Since all of the configuration on the client machine running upsmon that is relevant to connecting to the upsd server consists of a single line in a single file, I feel reasonably confident in assuming that the issue is with OPNsense and not the client. Since the error that I am receiving specifically mentions logging into the UPS is failing, it seems like the issue is with upsd users. OPNsense obfuscates its upsd.users file, which is actually located at /usr/local/opnsense/service/templates/OPNsense/Nut, and it is showing the following:

Code Select Expand

# Please don't modify this file as your changes might be overwritten with
# the next update.
#
{% if helpers.exists('OPNsense.Nut.general.enable') and OPNsense.Nut.general.enable == '1' %}
{%   if helpers.exists('OPNsense.Nut.general.mode') and OPNsense.Nut.general.mode == 'standalone' %}
{%     if helpers.exists('OPNsense.Nut.account.admin_password') and OPNsense.Nut.account.admin_password != '' %}
[admin]
password={{ OPNsense.Nut.account.admin_password }}
actions=set
instcmds=all
{%     endif %}
{%   endif %}
{%   if helpers.exists('OPNsense.Nut.general.mode') and OPNsense.Nut.general.mode == 'standalone' %}
{%     if helpers.exists('OPNsense.Nut.account.mon_password') and OPNsense.Nut.account.mon_password != '' %}
[monuser]
password={{ OPNsense.Nut.account.mon_password }}
upsmon master
{%     endif %}
{%   endif %}
{% endif %}

Based off of that, I have tried the following 4 variations on upsmon.conf on the FreeBSD client:

Code Select Expand

MONITOR cyberpower@OPNSENSE_IP:3493 1 monuser password2 slave

Code Select Expand

MONITOR cyberpower@OPNSENSE_IP:3493 1 monuser password2 master

Code Select Expand

MONITOR cyberpower@OPNSENSE_IP:3493 1 admin password1 master

Code Select Expand

MONITOR cyberpower@OPNSENSE_IP:3493 1 admin password1 slave

All of these also fail with the same error message, [ERR ACCESS-DENIED]. I have also tried restarting the nut service from within the OPNsense GUI while running upsmon -D on the FreeBSD client, which generates the following extra logs:

Code Select Expand

11.220715 Poll UPS [cyberpower@OPNSENSE_IP:3493] failed - Server disconnected
11.220735 Communications with UPS cyberpower@OPNSENSE_IP:3493 lost
13.346944 [D1] Trying to connect to UPS [cyberpower@OPNSENSE_IP:3493]
13.348476 Login on UPS [cyberpower@OPNSENSE_IP:3493] failed - got [ERR ACCESS-DENIED]
15.474333 Communications with UPS cyberpower@OPNSENSE_IP:3493 established

These logs imply that upsmon can definitely see the upsd server running on OPNsense, it just can't log into it.

I have tried changing the passwords from my initial complex ones to extremely simple ones with just 3 lowercase letters and am copying and pasting them from the OPNsense GUI to upsmon.conf, so I am certain that I am using the correct password and that it is a valid password.

Does anybody have any ideas? Is there anyone out there using this setup that could share their configuration for their NUT server on OPNsense and the upsmon.conf on their FreeBSD client?

[AbsolutelyFree]

So I "resolved" this, but in doing so I discovered two much bigger issues. Out of desperation, I manually edited the upsd.users file at /usr/local/etc/nut on OPNsense and manually wrote a user entry:

Code Select Expand


[test]
password=testing
upsmon secondary


I realize that this isn't the supported way of doing thing in OPNsense because it expects you to do everything via the UI and uses a custom version of upsd.users that is actually located at /usr/local/opnsense/service/templates/OPNsense/Nut that is derived from what is entered into the UI. Despite that, I reloaded the nut service and edited my upsmon.conf file on my FreeBSD server to read:

Code Select Expand


MONITOR cyberpower@OPNSENSE_IP:3493 1 test testing secondary


As soon as I ran upsmon -D, the server was immediately able to connect to the upsd server on OPNsense. This leads me to believe that the extra steps that OPNsense is taking to use a different upsd.users file that is generated from what users enter into the field in the UI for NUT is not correctly exposing the users that are created to the upsd service for the upsmon to connect to. Initially I was thinking that the issue was just with remote systems, but then I checked the system logs in OPNsense (System > Log Files > General) and I see that every single time the upsmon service is starting on OPNsense, it has the exact same issue authenticating with the upsd service:

Code Select Expand


2023-01-23T02:11:05-07:00 Error upsmon Login on UPS [cyberpower] failed - got [ERR ACCESS-DENIED]


This is saying that the upsmon service that is running on OPNsense isn't able to authenticate to the upsd service that is also running on OPNsense. It seems like the methods that OPNsense is using to take user input from the UI and enter them into custom files for the NUT service isn't working correctly, at least insofar as upsd.users. No upsmon services are able to connect to the upsd service, either local or remote, without manually editing the default files that NUT uses which you are told not to do with the comments in those files.



While investigating this issue, I went to unplug the UPS from my OPNsense router and plug it back in again to see what logs were generated. My OPNsense router is a passively cooled Protectli brand box. My hand brushed over the top of the box as I reached to unplug the USB cable and the box was so hot that it was painful to touch. I checked top on OPNsense and I saw that the upsd and upslog service are both using 60% of the CPU. This happens immediately as soon as the NUT service is started, even after I restart it. This is not normal, the NUT software is quite light.

Not sure what is happening here, but these are both significant issues with the NUT implementation in OPNsense. I am happy to provide any other information requested if anyone has ideas.