Upgraded from 26.1 today; Reporting/Traffic kernel panics the firewall

[gac]

I sent in a crash report about this, but the Reporting/Traffic page reproducibly panics my firewall every time I open it; the page loads with the empty graphs and then within 5 seconds everything is unresponsive and no longer routing traffic. It then spends multiple minutes (at least ~10) throwing out console messages over the serial port until I have to stop it and reboot it to make things work again.

- Barracuda F280 rev C
- AMD RX-421ND CPU
- multiple igb interfaces, mix of i211 copper and i350 fibre NICs

[franco]

Can you paste the backtrace from the panic here too for reference?


Thanks,
Franco

[gac]

I can - if I can find where it is stored, any clues? Now that I've submitted the crash report, it doesn't appear in the crash report window and I won't be able to trigger it again till this evening as I'm working from home and I don't know how long the firewall will actually be unavailable for since last night I gave up after ~10 minutes.

[franco]

It's gone after sending. Let me see if I can find it in the crash reports then.


Cheers,
Franco

[franco]

Yikes, this isn't good, but:

# sysctl net.bpf.zerocopy_enable
net.bpf.zerocopy_enable: 0

Do you have this turned on perhaps?


Cheers,
Franco

--

Code Select Expand

db:0:kdb.enter.default>  bt
Tracing pid 52686 tid 102576 td 0xfffff8000b7a6780
kdb_enter() at kdb_enter+0x33/frame 0xfffffe00667823a0
panic() at panic+0x43/frame 0xfffffe0066782400
trap_pfault() at trap_pfault+0x37d/frame 0xfffffe0066782450
calltrap() at calltrap+0x8/frame 0xfffffe0066782450
--- trap 0xc, rip = 0xffffffff8112ef70, rsp = 0xfffffe0066782520, rbp = 0xfffffe0066782520 ---
memmove_std() at memmove_std+0x30/frame 0xfffffe0066782520
bpf_zerocopy_append_mbuf() at bpf_zerocopy_append_mbuf+0x99/frame 0xfffffe0066782570
catchpacket() at catchpacket+0x5ed/frame 0xfffffe0066782620
bpf_mtap() at bpf_mtap+0x274/frame 0xfffffe00667826b0
vlan_transmit() at vlan_transmit+0x42/frame 0xfffffe0066782700
ether_output_frame() at ether_output_frame+0xd3/frame 0xfffffe0066782730
ether_output() at ether_output+0x6b0/frame 0xfffffe00667827c0
ip_output() at ip_output+0x1476/frame 0xfffffe00667828e0
tcp_default_output() at tcp_default_output+0x1f20/frame 0xfffffe0066782ab0
tcp_usr_ready() at tcp_usr_ready+0x182/frame 0xfffffe0066782b10
ktls_encrypt() at ktls_encrypt+0x2ba/frame 0xfffffe0066782d80
ktls_work_thread() at ktls_work_thread+0xfc7/frame 0xfffffe0066782ef0
fork_exit() at fork_exit+0x7b/frame 0xfffffe0066782f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0066782f30
--- trap 0xc, rip = 0x826db9c8a, rsp = 0x83f820f48, rbp = 0x83f820f60 ---

[gac]

Yes I do - not sure why as I don't run Suricata, but it looks like there are previous forum threads saying it might help Suricata performance. Perhaps I tried running Suricata, removed it, but never removed that tuneable.

[franco]

Okay disable should work but this definitely needs fixing in FreeBSD.


Cheers,
Franco

[gac]

Confirmed - disabling zerocopy (no reboot needed) does work, and the Reporting/Traffic page now works as it should. Thanks for the help!

[jonny5]

Came here to say I figured this out as well, took a second to be able to decompress the crash file and once I saw "zerocopy" in the list of commands/features, I was like oh yeah, I enabled that myself and it is a possibly useful but debated feature.

Disabled ("1" -> "0" for zerocopy) and everything worked - extra, but I had also uninstalled all microcode and realtek plug-ins, I have since re-installed the plug-ins, everything is working well.

Thank you OPNSense Community and Franco!! ^_^