Topics

Hi,

I am currently dealing with an annoying kernel panic on a Deciso DEC750.

Situation:
- Device was used for 12+ months behind a router "Digitalisierungsbox Premium 2", provided on a fiber business account by Deutsche Telekom - static IPv4 address and static /56 net + /64 address included. The router provided RFC1918 IPv4 addresses and NAT and set the Decisio Box as "exposed". Worked OK. (up to latest release of the 25.1 branch). Since the business router wouldn't provide DHCPv6 PD we decided to change the setup.

- The "Digitalisierungsbox" came with a plugged in "Digitalisierungsbox Smart 2" (aka Zyxel's GPON SFP module) which was then plugged in directly to the DEC750 on X0. After configuring PPPoE as usual the connection was established and work perfectly on IPv4 and IPv6, tracking the WAN for IPv6 subnets worked on all local VLANs. Firewall rules were extended to IPv6 and that's how it worked again, no problems detected.

- The OPNsense box also provides OpenVPN and IPv6 was working here as well. This includes OpenVPN access via IPv6 as well as handing out IPv6 addresses on OpenVPN.

- After a couple of days the box restarted itself after a kernel panic and repeated to do so every 2-7 days. Trying to trigger a panic by heavy load on traffic and/or OpenVPN was not successful. There is no indication on when the restart happens. Upgraded from the latest 25.1 to 25.7.3_7-amd64 and still it takes a couple of days for the next kernel panic to occur.

- Based on https://forum.opnsense.org/index.php?topic=41808.15 we tried disabling IPv6 on WAN and removed it from all VLANs and OpenVPN. And now the box runs smoothly as usual for at least 10 days now. No further kernel panic so far.

- The stack trace is like this:

Code Select Expand

panic: vm_fault_lookup: fault on nofault entry, addr: 0xfffffe003c242000
cpuid = 3
time = 1757468281
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00aa80c780
vpanic() at vpanic+0x131/frame 0xfffffe00aa80c8b0
panic() at panic+0x43/frame 0xfffffe00aa80c910
vm_fault() at vm_fault+0x15af/frame 0xfffffe00aa80ca30
vm_fault_trap() at vm_fault_trap+0x81/frame 0xfffffe00aa80ca80
trap_pfault() at trap_pfault+0x1be/frame 0xfffffe00aa80cad0
calltrap() at calltrap+0x8/frame 0xfffffe00aa80cad0
--- trap 0xc, rip = 0xffffffff80f7ec41, rsp = 0xfffffe00aa80cba0, rbp = 0xfffffe00aa80cbb0 ---
vm_radix_remove() at vm_radix_remove+0x51/frame 0xfffffe00aa80cbb0
vm_page_object_remove() at vm_page_object_remove+0x69/frame 0xfffffe00aa80cbd0
vm_page_free_prep() at vm_page_free_prep+0x24/frame 0xfffffe00aa80cbf0
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe00aa80cc20
vm_object_page_remove() at vm_object_page_remove+0x6a/frame 0xfffffe00aa80cc80
vm_map_entry_delete() at vm_map_entry_delete+0xf5/frame 0xfffffe00aa80ccc0
vm_map_delete() at vm_map_delete+0x7b/frame 0xfffffe00aa80cd30
vm_map_remove() at vm_map_remove+0x96/frame 0xfffffe00aa80cd60
vmspace_exit() at vmspace_exit+0xab/frame 0xfffffe00aa80cd90
exit1() at exit1+0x53a/frame 0xfffffe00aa80cdf0
sys_exit() at sys_exit+0xd/frame 0xfffffe00aa80ce00
amd64_syscall() at amd64_syscall+0x10e/frame 0xfffffe00aa80cf30

- The failure mode suggests some memory corruption that does not immediately show itself but causes a delayed kernel panic. So other than "switching off IPv6" I have no idea how to proceed right now. I checked system.log and did not find anything helpful or interesting other than a signal 11 on a python process

Code Select Expand


...
(repeating for some days until the crash)
2025-09-21T23:17:05 Notice dhcp6c dhcp6c_script: RENEW on pppoe0 executing
2025-09-21T23:02:05 Notice dhcp6c dhcp6c_script: RENEW on pppoe0 executing
2025-09-21T22:51:00 Notice kernel <6>[449403] pid 85765 (python3.11), jid 0, uid 0: exited on signal 11 (no core dump - bad address)
2025-09-21T22:47:05 Notice dhcp6c dhcp6c_script: RENEW on pppoe0 executing
2025-09-21T22:32:05 Notice dhcp6c dhcp6c_script: RENEW on pppoe0 executing
...



My questions:
- Is there anything I can do to locate the problem and may work around it? I would really like to write a bug report but just having a kernel panic in a page handler is not really helpful I guess.

- Can I do something about the DEC750, I am not an expert on this piece of hardware and it's properties and/or quirks (if any).

- Does anybody else have issues with that kind of setup and IPv6 and possibly a workaround?

Hi,

I am running out of ideas what to check with the following issue:

I have two instances of OPNSense, running on 25.1.7_4. One is within a proxmox VM and works fine. The other is my edge router (bare metal) and this is unable to handle new aliases.

What I did to exercise the problem:
1. Create new Alias "PC" (Host, 1 IPv4 LAN). Yes, clicked "Apply"!
2. Create a rule on LAN (Source "PC", Protocol enabled), pass. Yes, clicked "Apply"!
3. Trigger some traffic, nothing in the LiveView Log
4. Updated the rule using the verbatim IP address.
5. LiveView is showing a lot of traffic from the protocol rule.

Observations:
- In the alias section in firewall, the "last updated" column remains empty for "PC", load count is 0
- In the alias section in diagnostics, PC shows up as selectable item but shows no contents.
- Global configuration in /conf/config.xml contains the alias definition
- Checked /var/db/aliastables, no entry for "PC" - the filesystem has plenty of space left and permissions seem ok
- Checked backend log: Nothing of a warning or higher severity, nothing relevant (from my perspective) in less severe levels.
- Checked firewall log: No warning or higher, nothing about alias (had to search for the term "alias")
- Cloudflare, Spamhaus DROP and GeoIP seem to regenerate  as usual, timestamp of /var/db/aliastables matches log entries

The only "interesting" part about this machine is that I replaced the SSD 4 weeks ago, ran a full install and reloaded the last known config / backup. Updated to 24.1.7_4 in the process afterwards.

I know I can stick to hard coded IP addresses for now - and I will not reboot until the next weekend at least, so testing it is currently not possible. My second instance on Proxmox does not have this issue and updates everything as required.

EDIT: (See reply below for more) running configctl filter refresh_aliases returned no output other than an empty line.

Are there any other locations I might have a look for diagnostics or trigger an alias re-generation from the shell?

Thanks.

EDIT2/Resolution: flock was blocking forever on a lock existing for more than 21 days. I'd expect however the firewall to not silently do nothing in such a case.

Hi,

some of the major glitches w.r.t. IPv6 seemed to have broken android connectivity via wifi. Seems to work with 24.7.3 again.

Was investigating since 24.7.2 because some androids in my network were dropping wifi every now and then. The wifi is provided by unifi UAPs (no gateway). Windows and Linux PCs had no issue. Androids via wired LAN were also fine. I suspect a possible issue with IPv6 that caused the androids to think that wifi has no internet connection and drop it. Unifi firmware downgrades would not have any beneficial effect and could be ruled out.

I also noticed some reporting by OPNsense that fe80:: wants to talk to 2a00::. I don't know if that's relevant to my issues or just an effect during Android's connection drops.

Nevertheless, androids are now online without frequent drops. 24.7.3 fixed it. Hope that helps others with the same issue.

[My Expectation 1: [passed]]

Hi,

I have an issue understanding something, however I must admit that my expectations might be wrong.

Test setup is:

• OPNSense Box 1 (Router 1) has LAN 10.0.1.1/24, WAN is public ISP provided, static IP
• OPNSense Box 2 (Router 2) has WAN 10.0.1.99 and LAN 10.0.64.1/24. Router2's WAN is in fact connected to the router 1's LAN network.
  
• Router 1 does not know about 10.0.64.0/24, no route to that network configured.
• Router 2 is configured statically on WAN and LAN, no DHCP Client involved on WAN. Configured 10.0.1.1 as default upstream gateway. Router 2 uses outbound NAT.

My Expectation 1: [passed]
TCP to public internet or services in Router 1's LAN are successful from Router 2's LAN. OPNsense outputs traffic to Router 1's LAN without the gatway via layer 2

My Expectation 2: [failed]
I can enable port forwarding on Router 2 to allow services from behind Router 2 to be exposed to Router 1's LAN.

So, I created a port forwarding and allowed an associated firewall rule. Observation: No access to exposed service via forwarded port from clients in Router 1's LAN 10.0.1.0/24.

Observing the live view in both OPNsenses it turned out that

• first the client in 10.0.1.0/24 connects to the forwarded port and the traffic is forwarded correctly.
  
• answers are sent to the default GW of Router 2, i.e. Router 1 which issues a state rule violation in live traffic view
• After disabling the default GW, it works as expected, traffic goes directly back to the client via layer 2

I would have thought that the default GW should not be part of the equation no matter if I just use outbound NAT or port forwarding. The destination IP is in the WAN networks range and should not require a gateway. Did I miss something?

Hi,

I observed some unexpected behaviour in dpinger / OPNsense's handling of dpinger tonight. At least to me it is unexpected, so correct me if I am wrong.

Status:
OPNSense 23.7.5-amd64 on Zotac's ZBOX with 2x 1 GBits Realteak NICs and os-realteak-re installed. Works stable for more than 4 years. WAN on re0 (no VLAN) is directly connected to a Mikrotik router provided from the ISP, so I don't have access to it.

Event:
Due to a reboot of the mikrotik there was an IF DOWN/UP at 04:19:16 - 04:19-26 and again at 04:19:45 - 04:19:49.

Observation:
1. OPNSense detected IF events and reacted accordingly. See system.log below.
2. DPinger noticed that it failed and was restarted twice. When I checked the system this morning at 10:00 I saw that dpinger for IPv6 was down - so was the gateway as shown in OPNSense's dashboard. I triggered a start of dpinger for the IPv6 gateway on the dashboard and it started successfully. The gateway became online and IPv6 was working again.

My expectation:
I would have expected dpinger for IPv6 to resume to normal operation after the incident and not to remain in STOPPED state.

Is this really intended behaviour or do I have to adjust some options for gateway monitoring?


gateways.log for the affected time:

2023-10-04T10:01:43   Warning   dpinger    send_interval 1000ms  loss_interval 4000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 0ms  loss_alarm 0%  alarm_hold 10000ms  dest_addr 2a00:xxxx:xxxx:xx::1  bind_addr 2a00:xxxx:xxxx:xx::2  identifier "ISP_WAN_GWv6 "
2023-10-04T04:19:50   Warning   dpinger    send_interval 1000ms  loss_interval 4000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 0ms  loss_alarm 0%  alarm_hold 10000ms  dest_addr xx.xx.148.97  bind_addr xx.xx.148.98  identifier "ISP_WAN_GWv4 "
2023-10-04T04:19:50   Warning   dpinger    exiting on signal 15
2023-10-04T04:19:49   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:48   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:47   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:46   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:45   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:44   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:43   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:42   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:41   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:40   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:39   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:38   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:37   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:36   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:35   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:34   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:33   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:32   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 64
2023-10-04T04:19:27   Warning   dpinger    send_interval 1000ms  loss_interval 4000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 0ms  loss_alarm 0%  alarm_hold 10000ms  dest_addr xx.xx.148.97  bind_addr xx.xx.148.98  identifier "ISP_WAN_GWv4 "
2023-10-04T04:19:27   Warning   dpinger    exiting on signal 15
2023-10-04T04:19:26   Warning   dpinger    exiting on signal 15
2023-10-04T04:19:26   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 55
2023-10-04T04:19:25   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:25   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:24   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:24   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:23   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:23   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:22   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:22   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:21   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:21   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:20   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:20   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:19   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:19   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:18   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:18   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-10-04T04:19:17   Warning   dpinger    ISP_WAN_GWv4 xx.xx.148.97: sendto error: 65
2023-10-04T04:19:17   Warning   dpinger    ISP_WAN_GWv6 2a00:xxxx:xxxx:xx::1: sendto error: 50
2023-09-28T08:56:32   Warning   dpinger    send_interval 1000ms  loss_interval 4000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 0ms  loss_alarm 0%  alarm_hold 10000ms  dest_addr xx.xx.148.97  bind_addr xx.xx.148.98  identifier "ISP_WAN_GWv4 "
2023-09-28T08:56:32   Warning   dpinger    send_interval 1000ms  loss_interval 4000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 0ms  loss_alarm 0%  alarm_hold 10000ms  dest_addr 2a00:xxxx:xxxx:xx::1  bind_addr 2a00:xxxx:xxxx:xx::2  identifier "ISP_WAN_GWv6 "


system.log for the affected time

2023-10-04T04:20:15   Warning   opnsense    /usr/local/etc/rc.linkup: warning: ignoring missing default tunable request: debug.pfftpproxy
2023-10-04T04:20:14   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-10-04T04:20:14   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-10-04T04:20:14   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,wan))
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure ipsec (,wan)
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (execute task : dpinger_configure_do(,ISP_WAN_GWv4))
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (,ISP_WAN_GWv4)
2023-10-04T04:19:50   Warning   opnsense    /usr/local/etc/rc.linkup: The required ISP_WAN_GWv6 IPv6 interface address could not be found, skipping.
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (execute task : dpinger_configure_do(,ISP_WAN_GWv6))
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (,ISP_WAN_GWv6)
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: keeping inet6 default route to 2a00:xxxx:xxxx:xx::1
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: configuring inet6 default gateway on wan
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: setting inet default route to xx.xx.148.97
2023-10-04T04:19:50   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: configuring inet default gateway on wan
2023-10-04T04:19:49   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'
2023-10-04T04:19:49   Notice   opnsense    /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for wan(re0)
2023-10-04T04:19:49   Notice   kernel    <6>re0: link state changed to UP
2023-10-04T04:19:46   Notice   opnsense    /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for wan(re0)
2023-10-04T04:19:45   Notice   kernel    <6>re0: link state changed to DOWN
2023-10-04T04:19:32   Warning   opnsense    /usr/local/etc/rc.linkup: warning: ignoring missing default tunable request: debug.pfftpproxy
2023-10-04T04:19:32   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-10-04T04:19:32   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-10-04T04:19:32   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-10-04T04:19:27   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-10-04T04:19:27   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-10-04T04:19:27   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,wan))
2023-10-04T04:19:27   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure ipsec (,wan)
2023-10-04T04:19:27   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (execute task : dpinger_configure_do(,ISP_WAN_GWv4))
2023-10-04T04:19:27   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (,ISP_WAN_GWv4)
2023-10-04T04:19:27   Warning   opnsense    /usr/local/etc/rc.linkup: The required ISP_WAN_GWv6 IPv6 interface address could not be found, skipping.
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (execute task : dpinger_configure_do(,ISP_WAN_GWv6))
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: plugins_configure monitor (,ISP_WAN_GWv6)
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: keeping inet6 default route to 2a00:xxxx:xxxx:xx::1
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: configuring inet6 default gateway on wan
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: setting inet default route to xx.xx.148.97
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: configuring inet default gateway on wan
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'
2023-10-04T04:19:26   Notice   opnsense    /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for wan(re0)
2023-10-04T04:19:26   Notice   kernel    <6>re0: link state changed to UP
2023-10-04T04:19:17   Notice   opnsense    /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for wan(re0)
2023-10-04T04:19:16   Notice   kernel    <6>re0: link state changed to DOWN