Messages

I dug around a bit... If you create a Monit task to check for the service, you can at least start it automagically – but this is only a workaround.

So if anybody else is affected by this issue (there are not many RMM tools out there supporting BSD), the Monit task settings:


Type: Process
Match: pulsewayd
Start: /etc/rc.d/pulseway start
Stop: /etc/rc.d/pulseway stop
Tests: Nothing selected
Depends: Nothing selected


Hope this helps others until a better solution becomes available :-)

Hi there,

here's an issue which seems related to 25.7.8. If the Pulseway RMM (or Kaseya VSA) is installed on an OPNsense system, it'll fail starting the service on boot up. Service is started by rc.conf.

25.7.8 was installed on one system only so far.

Code Select Expand

<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="324"] <118>[26] Starting pulseway.
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 73890 - [meta sequenceId="325"] /usr/sbin/service: WARNING: $mpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 78128 - [meta sequenceId="326"] /usr/sbin/service: WARNING: $dhcp6c_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="327"] <118>[26] Starting lldpd.
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="328"] <118>[26] pulseway already running?  (pid=22825).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="329"] <118>[26] >>> Invoking start script 'syslog'
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="330"] <118>[26] >>> Invoking start script 'xen'
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="331"] <118>[26] Starting xenguest.
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="332"] <118>[27] >>> Invoking start script 'carp'
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="333"] <118>[27] >>> Invoking start script 'cron'
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 90361 - [meta sequenceId="334"] /usr/sbin/service: WARNING: $unbound_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 93123 - [meta sequenceId="335"] /usr/sbin/service: WARNING: $radvd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 37088 - [meta sequenceId="336"] /usr/sbin/service: WARNING: $configd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 49328 - [meta sequenceId="337"] /usr/sbin/service: WARNING: $openvpn_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 50789 - [meta sequenceId="338"] /usr/sbin/service: WARNING: $openssh_enable is not set properly - see rc.conf(5).
<45>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain syslog-ng 13690 - [meta sequenceId="1"] Configuration reload request received, reloading configuration;
<45>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain syslog-ng 13690 - [meta sequenceId="2"] Configuration reload finished;
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 59564 - [meta sequenceId="3"] /usr/sbin/service: WARNING: $dhcpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 61028 - [meta sequenceId="4"] /usr/sbin/service: WARNING: $dhcpd6_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 62616 - [meta sequenceId="5"] /usr/sbin/service: WARNING: $strongswan_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 65911 - [meta sequenceId="6"] /usr/sbin/service: WARNING: $snmptrapd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 67926 - [meta sequenceId="7"] /usr/sbin/service: WARNING: $snmpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 70971 - [meta sequenceId="8"] /usr/sbin/service: WARNING: $rrdcached_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 83400 - [meta sequenceId="9"] /usr/sbin/service: WARNING: $kpropd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 86704 - [meta sequenceId="10"] /usr/sbin/service: WARNING: $samplicator_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 88919 - [meta sequenceId="11"] /usr/sbin/service: WARNING: $xenguest_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 91219 - [meta sequenceId="12"] /usr/sbin/service: WARNING: $choparp_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 93550 - [meta sequenceId="13"] /usr/sbin/service: WARNING: $lighttpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 97460 - [meta sequenceId="14"] /usr/sbin/service: WARNING: $php_fpm_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="15"] OK
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 39706 - [meta sequenceId="16"] /usr/sbin/service: WARNING: $mpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 42364 - [meta sequenceId="17"] /usr/sbin/service: WARNING: $dhcp6c_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 55528 - [meta sequenceId="18"] /usr/sbin/service: WARNING: $unbound_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain root 57465 - [meta sequenceId="19"] /usr/sbin/service: WARNING: $radvd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain opnsense 59989 - [meta sequenceId="20"] /usr/local/sbin/pluginctl: plugins_configure crl (1)
<13>1 2025-11-28T16:33:26+01:00 OPNsensePlayground.localdomain opnsense 59989 - [meta sequenceId="21"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : core_trust_crl(1))
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 84561 - [meta sequenceId="22"] /usr/sbin/service: WARNING: $configd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 93146 - [meta sequenceId="23"] /usr/sbin/service: WARNING: $openvpn_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 95159 - [meta sequenceId="24"] /usr/sbin/service: WARNING: $openssh_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 1440 - [meta sequenceId="25"] /usr/sbin/service: WARNING: $dhcpd_enable is not set properly - see rc.conf(5).
<12>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain opnsense 59903 - [meta sequenceId="26"] /usr/local/sbin/pluginctl: warning: ignoring missing default tunable request: vm.pmap.pti
<12>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain opnsense 59903 - [meta sequenceId="27"] /usr/local/sbin/pluginctl: warning: ignoring missing default tunable request: hw.ibrs_disable
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 3069 - [meta sequenceId="28"] /usr/sbin/service: WARNING: $dhcpd6_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 5221 - [meta sequenceId="29"] /usr/sbin/service: WARNING: $strongswan_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 7231 - [meta sequenceId="30"] /usr/sbin/service: WARNING: $snmptrapd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 9190 - [meta sequenceId="31"] /usr/sbin/service: WARNING: $snmpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 11137 - [meta sequenceId="32"] /usr/sbin/service: WARNING: $rrdcached_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="33"] <118>[27] >>> Invoking start script 'openvpn'
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="34"] <118>[27] >>> Invoking start script 'sysctl'
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain opnsense 59989 - [meta sequenceId="35"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : openvpn_refresh_crls(1))
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 21310 - [meta sequenceId="36"] /usr/sbin/service: WARNING: $kpropd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 22332 - [meta sequenceId="37"] /usr/sbin/service: WARNING: $samplicator_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 23532 - [meta sequenceId="38"] /usr/sbin/service: WARNING: $xenguest_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 24823 - [meta sequenceId="39"] /usr/sbin/service: WARNING: $choparp_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 26010 - [meta sequenceId="40"] /usr/sbin/service: WARNING: $lighttpd_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain root 27649 - [meta sequenceId="41"] /usr/sbin/service: WARNING: $php_fpm_enable is not set properly - see rc.conf(5).
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="42"] <118>[27] Service `sysctl' has been restarted.
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="43"] <118>[27] >>> Invoking start script 'beep'
<11>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain pulsewayd 22825 - [meta sequenceId="44"] Signal handled: 11
<13>1 2025-11-28T16:33:27+01:00 OPNsensePlayground.localdomain kernel - - [meta sequenceId="45"] <6>[28] pid 22825 (pulsewayd), jid 0, uid 0: exited on signal 11 (no core dump - bad address)

If the Pulseway service is started after OPNsense is fully stated up, it starts and runs:

Code Select Expand

root@OPNsensePlayground:~ # service pulseway start
Starting pulseway.
root@OPNsensePlayground:~ # service pulseway status
pulseway is running as pid 77088.

If the system is reverted back to 25.7.7_4, everything works again and the service is started during boot up.


I'm posting it here because it might not be Pulseway's fault because the service does start fine manually. And the release notes for 28.7.8 said we should report weird behaviour which might be related to the newer kernel.

The dual stack stuff needs more overhead and therefore reduces the usable MTU size. ry reducing it to 1300 or even smaller. DS-lite and non working VPN connections are a very common issue. Reducing MTU is the best bet to solve these issues.

ZeroTier modifies the Ethernet address of the device on its own.

It has to. Each device in a ZT network has its own MAC which is calculated from the member id of that device. This address does not change as long as the member id doesn't change which it only does if someone manually resets the member id and therefore makes it a new device to ZT.

ZT needs its own MAC because it works as a SDWAN switch and needs arp to function.

That answers the question why nothing arrived at the firewall. It was just impossible to send Ethernet frames to the ZT network member MAC from OPNsense.

I have no way of testing this

I can set up a public ZT network for you to play with, just drop me a line

so someone with the setup please take a closer look at ifconfig in the working and non-working case.

Just the ZT part:

Non working:

Code Select Expand


REDACTED: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 5000 mtu 2800
description: ZeroTier (opt2)
options=80000<LINKSTATE>
ether 58:9c:fc:10:92:2f
inet 172.27.8.25 netmask 0xffff0000 broadcast 172.27.255.255
inet6 fe80::5a9c:ffff:ffff:ffff%REDACTED prefixlen 64 scopeid 0x7
groups: tap
media: Ethernet 1000baseT <full-duplex>
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Opened by PID 66352



Working after applying the patch:

Code Select Expand


REDACTED: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 5000 mtu 2800
description: ZeroTier (opt2)
options=80000<LINKSTATE>
ether 7a:fd:ba:es:1f:1c
hwaddr 58:9c:fc:10:92:2f
inet 172.27.8.25 netmask 0xffff0000 broadcast 172.27.255.255
inet6 fe80::5a9c:ffff:ffff:ffff%REDACTED prefixlen 64 scopeid 0x7
groups: tap
media: Ethernet 1000baseT <full-duplex>
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Opened by PID 61635


My assumption is still that this is true for assigned ZeroTier interfaces, but maybe I missed someone confirming that.

The ZT networks are assigned to an interface in my case, yes.

And is this an IPv4 or IPv6 tunnel?

IPv4 in the tunnel

# opnsense-patch 1dba25fed8

Applied to 24.7.4 and ZT is working again after a reboot.

> Reverting to 24.7.3_1 restores ZT connectivity immediately.

Someone will tell me what this means? Full revert? Still leaves the question if this is a kernel or core issue...

It means the whole system was reverted to 24.7.3_1.

There have been issues in the past, mostly because of routing issues within OPNsense – I guess – in all cases the device was connected properly to ZT was able to see all neighbours. But no comms to OPNsense. A zt leave and zt join fixed it (or removing the checkmark for the network in the gui).

This time it's different. Everything works with 24.7.3_1 (and with many releases prior, too. The above issue was last seen sometimes earlier this year). After installing 24.7.4 ZT no longer works.

A zt leave and zt join don't solve it anymore, no comms.

The firewall rules to allow data flow from the ZT network to other networks or the firewall itself don't show any states in inspect mode.

Reverting to 24.7.3_1 restores ZT connectivity immediately.

Question is: how to hunt down "which change, which component update of this affects ZeroTier operation"? Any directions on where to start?

Ah. My mistake. I got a bit confused because it looks different from 24.1 and it shows less information in the list view. And it seemed as if the remove did nothing. It does.

Remove works. No errors :-)

Could it be that this didn't make it into the 24.7.3 (or not fully)?

I tried an API request and the response from trust/crl/search doesn't contain a UUID like cert/search or ca/search.

So I assume that the UI doesn't get a UUID, too and therefore cannot remove the CRL entry.

Deleting a revocation under "System: Trust: Revocation" trows error "Endpoint not found". Happens in 24.7_x and 24.7.1.

Certs have been revoked sometimes with 24.1 or 23.x – so it might be something that happens to older entries only. Have no newer revocations.

Putting two 10/1 connections on one tier will *not* create one single connection out of both. They are still completely separate. The router/firewall just distributes connections across both. It's no aggregations, this would need some support on the remote side to enable such feature.

I would not expect a full 20/2 in this scenario.

You can pull one WAN down and check the speed again, then repeat with the other WAN down.

But don't forget that PPPoE has an absurdly high overhead which will significantly increase the system load. So depending on the used hardware, the system might as well reach some bottlenecks.

Regarding your IPTV, it's normally never using both connections at the same time and is only routed via one WAN (because the packets cannot be combined without the previous mentioned aggregation support on the ISP side).

This behavior can not be turned off and I suspect it is the same on XenServer.

Maybe I completely misunderstood the problem here but sure... you can not only change the MAC to your liking, the MAC is fixed.

I've attached a screenshot from XenOrchestra.

Hi!

Maybe I'm mistaken but... Sending a request to API endpoint kea/dhcpv4/getSubnet responds with subnet details like DNS servers, routers and NTP. But the subnets configured are not part of the response.

I guess the field "subnet" should contain the CIDR, but it doesn't.

Not sure if this is a known issue, I couldn't find anything in the forum.

Screenshots attached.

OPNsense version is 24.1.2_1

And a small addition: the pools section is empty, too. And if there are more than one subnet defined, only the first one is in the API response.

Take your time, no need to hurry :-)

I installed the patch and the lease duplicates are gone.