Messages

This was the root cause of my issue as well. Annoying that it was something so simple and yet I did not think of it!!

I am still having this issue, and I have delved a bit deeper. I can see that when the /ui/core/hasync_status page is loaded, it makes an API call to /api/core/hasync_status/version. On my firewalls (2 separate pairs in different networks) this API endpoint responds:

Code Select Expand

{
  "status": "error",
  "message": "parse error. not well formed"
}
I have seen reference to this error in old bugs around HA sync, for example
https://forum.opnsense.org/index.php?topic=20557.0
https://github.com/opnsense/core/issues/4533

A different bug but similar symptoms. I think it means that the IXR library cannot parse the config file, as in, it is not well formed XML. However unticking all of the different services to synchronise in the HA Settings page does not make a difference to me. Either way, the error message displayed in the web GUI is clearly unrelated to the error returned by the API, and is quite misleading!

How can I work out what bit of the config/HA setup IXR is saying is not well formed?

I have the same issue on 2 business edition firewalls. They were upgraded to 25.4 last week and since then the primary firewall displays "backup firewall unavailable (check user creds)". HA sync worked prior to the upgrade - I did one the same day, before upgrading. I have manually reset all passwords on both firewalls and it's made no difference, and as per the OP the rc.filter_synchronize script from the command line works fine.

tcpdump on either firewall (tcpdump -i igb0 port 443) shows no packets when clicking System -> High Availability -> Status, however plenty of PFSYNC traffic is visible when running tcpdump without the port specification. I can curl https://ha-peer-ip-address/ with no issues from the CLI.

CVE-2025-27516 affecting Jinja2 was fixed in community yesterday and isn't much older than that if at all exploitable. I already planned to hotfix business, but we also need to ensure that these things don't cause regressions first. But also:

plugins % git grep '|[^a-z]*attr' */*/src/opnsense/service/templates | wc -l
       0

core % git grep '|[^a-z]*attr' src/opnsense/service/templates | wc -l
       0

For CVE-2025-26466 it's a bit different. Medium score and DoS warrant patching and I agree it needs patching in the next release, though that's also where it would be patched at the latest anyway.  By default SSH is not exposed and you can even use IPS or firewall to rate limit.


Cheers,
Franco

Thank you Franco, that is very useful ammunition for me to go back with around Jinja2. Is there a way to pull the latest ssh package from ports while leaving the rest of opnsense alone?

Vulnerability scanners are blunt instruments, and context always matters. Your security team should adjust for that.

Based on your description, you already had the mitigations in place before either vulnerability was announced.

Priority wise, should you actually have an attacker on the management network there is an argument to be made that higher value targets exist there. Bringing down a bunch of FWs would hardly be a financially rewarding endeavour.

Yes, I am part of the security team :) The issue I am trying to resolve is really one of upward management, because reports from vulnerability scanners go directly to the customer in this environment which results in pressure. That is not an OPNsense problem though so wasn't going to go through all that detail on here, it is a customer management issue. Of course several technical mitigations are in place that significantly reduce the exposure.

We run an environment with several business edition Deciso hardware OPNsense firewalls. There are strong compliance and vulnerability requirements in the environment, as it is CNI adjacent. SLA requires vulnerabilities to be mitigated or patched within days not weeks, which goes against the patching ethos of business edition.

At the time of writing, there is CVE-2025-26466 affecting OpenSSH < 9.9p2 and CVE-2025-27516 affecting Jinja2. Is there a (supported?) way the updated versions of these packages can be pulled and installed to satisfy my SLA requirements around timely remediation of vulnerabilities? SSH just comes from ports (I think?) so I think that could be trivial, not so sure about Jinja2. The exposure is obviously reduced as these services are not exposed anywhere other than an internal management network, but that doesn't stop the vulnerability scanner sending big red warnings to the security team...

I've made a load of improvements to a plugin but have hit a blocker. The plugin in question has to write the IP address for a service to listen on, into a config file. I can use the templates structure for the plugins, e.g.

Code Select Expand

{%     if helpers.exists('interfaces.'+int+'.ipaddr') %}
{%       set interface_ip = helpers.getNodeByTag('interfaces.'+int+'.ipaddr') %}
{%       if '.' in interface_ip %}
--- do stuff here ---
{%       endif %}
{%     endif %}


However that gets the IP address out of the config file. If an interface is set to DHCP, or SLAAC (for the ipaddrv6 property) there is no IP address in that field. How do we cope with that in a plugin, is there an event handler for an IP address change that can update config files for running services?

Actually, I've had a bit of a think about this. What I will do (if it's OK with you) is fork the snmp plugin and update that with what I've done here instead. That way we aren't building very specific solutions for very specific products into the body of the main OS - modifying system components for the sake of one 3rd party product. Instead I will roll what I've written here into the snmp plugin.

That would be awesome!

Minimal mucking about:

Code Select Expand

./opnsense-version -o
FreeBSD|SMP|amd64|OPNsense|24.7.8|vmware|

That should mean you could add something into your snmpd.conf template like:

Code Select Expand

{% if OPNsense.netsnmp.general.enableobservium == '1' %}
extend .1.3.6.1.4.1.2021.7890.1 distro /usr/local/sbin/opnsense-version -o
extend .1.3.6.1.4.1.2021.7890.2 hardware /bin/kenv smbios.planar.product
extend .1.3.6.1.4.1.2021.7890.3 vendor /bin/kenv smbios.planar.maker
extend .1.3.6.1.4.1.2021.7890.4 serial /bin/kenv smbios.planar.serial
{% endif %}

And then not have to bundle the distro.sh from Observium. It's actually better than distro.sh which will never detect if you are running in a hypervisor - despite having logic to check if it is running in FreeBSD, it then tries to call the detectvirt binary which is a systemd tool that doesn't exist here.  My virtualisation detection is rudimentary though - it doesn't detect containers and I can only actually test it on vmware and hyper-v.

I will put this in a PR and see what Franco et al think  :)

Just from a quick view, the format Observium is expecting for OS info is:

${OS}|${KERNEL}|${ARCH}|${DISTRO}|${VERSION}|${VIRT}|${CONT}

We could easily modify the built-in opnsense-version script to add another parameter (say, -o) to output Observium format. You would then not need distro.sh which contains a large amount of irrelevant info (e.g. checking to see if we are running on Solaris). Then you haven't got to maintain Observium's distro.sh script inside an OPNsense plugin, bundle in the Observium license, etc etc.

I understand now. That revision is far too new to be on my production systems I'm afraid - most are business edition! I will investigate it though because I would be keen to do this in such a way that would work any any NMS rather than specific 3rd party scripts for 1 product - e.g. some (most?) of the data from distro.sh is already provided by the net-snmp plugin in this other OID which is referenced by other 3rd party tools. And as mentioned I also have a bunch of work on the os-net-snmp and os-lldpd plugins in flight  :)

Ah, no sorry I didn't see that bit. I've not installed that plugin because I'm not using Observium, however it is interesting they have chosen to do it that way, as the info about architecture, release, and OS name (e.g. the stuff I posted about above) is exposed by net-snmpd by ticking the 'Display Version in OID' tickbox without needing any additional plugins.

What else does that plugin do? It can't just be that! Off to investigate... shouldn't need to install a plugin to make an nms work

The net-snmpd plugin has an option to expose the version of OPNsense under a specific OID. This is a tickbox in the GUI provided by this plugin:



This adds the following line to the snmpd config file:

Code Select Expand

extend    version   /usr/local/sbin/opnsense-version


This has the effect of putting whatever the output is of that command, into that OID.

Observium uses .1.3.6.1.4.1.2021.7890.1 which is what you get when you enable Observium support in the current version of the plugin. Just for reference - I have no idea why there are (at least) two different OIDs. Also Observium does not use opnsense-version but relies on its own "distro" script for all Unix platforms.

What does it report in that OID? I don't get anything on my OPNsense querying that:

Code Select Expand

root@x-y-z:~ # snmpwalk -v3 -u xxx -a sha -x aes -A xxx -X xxx -l authPriv 127.0.0.1 .1.3.6.1.4.1.2021.7890.1
UCD-SNMP-MIB::ucdavis.7890.1 = No Such Object available on this agent at this OID


[think]

Is from updates, not a fresh install. Literally upgraded to 24.10_7 last night and it happened immediately.

I had HTTP_PROXY and HTTPS_PROXY set in the configd template and neither the gui nor update-crl-fetch.py worked. Adding http_proxy and https_proxy in the configd template made update-crl-fetch.py work, but the gui still not. Removing the file from /tmp made no difference - it would just be recreated. I ticked the Auto add CRLs tickbox and went to sleep, and when I woke up it was fixed.

I think that when I looked at the contents of the file, it just contained:

Code Select Expand

# [i] fetch certificate for https://opnsense-update.deciso.com

Happy to, I have a few more for net-snmpd / lldpd waiting in the wings too! The question I guess is, do any changes that have been made in the last 5 years now rely on the new default output of the command?