I'd appreciate some help with this please.
I recently started using traffic shaping and I'm getting a periodic fatal trap and reboot of my OPNSense box (see below). It's running on a Protectli box with Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (4 cores)
Does anyone have any suggestions for tracking this down and resolving it?
This seems to occur only when traffic shaper rules are enabled. As a test, I disabled the traffic shaper rules and the firewall has run for 14 hours without rebooting. Enabling traffic shaper rules results in a reboot within 30m-3 hours.
Here is the version info off the dashboard:
OPNsense 20.1.7-amd64
FreeBSD 11.2-RELEASE-p20-HBSD
OpenSSL 1.1.1g 21 Apr 2020
-----------------------------------
Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 02
fault virtual address = 0xffffffff00040061
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80d9c2e3
stack pointer = 0x28:0xfffffe0232b352a0
frame pointer = 0x28:0xfffffe0232b352a0
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0xffffffff00040061
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80daa0b3
stack pointer = 0x28:0xfffffe0232dee7a0
frame pointer = 0x28:0xfffffe0232dee7f0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 0 (dummynet)
version.txt0600007013663554527 7550 ustarrootwheelFreeBSD 11.2-RELEASE-p20-HBSD 07ef86ce9ca(stable/20.1)
UPDATE: Resolved.
After backing up the configuration, then re-installing OPNsense and restoring the configuration, my OPNsense box has not encountered a crash/random reboot with traffic shaping enabled. The re-install installed 20.1 then upgraded to 20.1.7. I've been running for over 14 hours without a crash.
Note: Before re-installing, I had performed a Firmware audit several times and everything checked out, but I was still encountering periodic crashes and reboots only when traffic shaping was enabled.
I also tried re-installing key packages including opnsense, but I continued to encounter random crashes and reboots only when traffic shaping was enabled.
Conclusion: Successful completion of firmware audit does not provide a comprehensive check of the integrity of the OPNSense install.
Do you use latest 20.1.7?
Here is the version info off the dashboard:
OPNsense 20.1.7-amd64
FreeBSD 11.2-RELEASE-p20-HBSD
OpenSSL 1.1.1g 21 Apr 2020
I believe this is related to traffic shaping. If I disable all the shaper rules, I don't see the periodic reboots (uptime of the firewall is 14hours and counting). If I enable the traffic shaper rules, I'll get the periodic reboots happening again (uptime <2 hours before a reboot).
Is anyone actively using traffic shaping in OPNsense 20.1.7-amd64, and NOT getting random reboots?
I use traffic shaper but only as a throttle to help with buffer bloat. Nothing fancy. No reboots.
My shaper also runs fine since update to 20.1.7 (5 days ago) ...
For people who are running traffic shaping without periodic crashes/reboots, what hardware are you running OPNsense on? I am seeing the issues on an I5-7200U
Do you have PIE active queue management enabled on any of your pipes or queues? I have experienced stability problems (seemed to be a complete lockup, but not a reboot) with PIE enabled before, but everything works fine when I disable PIE and use CoDel.
I am only using FlowQueue-CoDel.
On my i5-7200U hardware, I performed a health audit and even reinstalled key packages (like the core OPNsense package), but I'm still getting random reboots.
As a test, I installed OPNsense on a box with a Celeron J3455 (also with Intel NICs), and restored the config from the i5-7200U box. So far, it has run for 14 hours without a random reboot.
I am currently running memtest on the i5-7200U box--so far no errors. Then I'll try a fresh install of OPNSense on that box & restore the config to that box as a test.