1
24.7 Production Series / Router crashes once a week, Fatal trap 12: page fault while in kernel mode
« on: September 25, 2024, 06:26:46 am »
Once a week my router will crash saying fatal trap 12. I've run memtest86 on the hardware so I know CPU and RAM is good. I've also swapped out my RAM for a new stick and it's still crashing. Dump below:
System Information:
dmesg.boot: (cut for brevity)
/var/crash/info.0:
/var/crash/info.1:
/var/crash/info.2:
/var/crash/info.3:
/var/crash/info.4:
/var/crash/textdump.tar.0 can be provided on request
System Information:
Code: [Select]
User-Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36
FreeBSD 14.1-RELEASE-p4 stable/24.7-n267825-d0d18dbbaba SMP amd64
OPNsense 24.7.4_1 b7c86e348
Plugins os-iperf-1.0_1 os-sensei-1.17.6 os-sensei-agent-1.17.5 os-sensei-updater-1.17 os-sunnyvalley-1.4_3
Time Tue, 24 Sep 2024 21:17:39 -0700
OpenSSL 3.0.15
Python 3.11.9
PHP 8.2.23
dmesg.boot: (cut for brevity)
Code: [Select]
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e75, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e78, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e75, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e78, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:9000:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:f400:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:2600:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:ea00:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:c200:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:d400:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:8600:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:3600:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e78, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e75, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:e200:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:e00:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:9000:24bb:c600:14:e9d0:1740:21, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e75, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e78, nxt 6, rcvif igc0, outif igc1
cannot forward src fe80:1::ce6e:a4ff:fef9:90e9, dst 2600:1406:3a00:1f::173e:2e78, nxt 6, rcvif igc0, outif igc1
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 08
fault virtual address = 0x20
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80bd0d8f
stack pointer = 0x28:0xfffffe00846abe20
frame pointer = 0x28:0xfffffe00846abec0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 2 (clock (0))
rdi: fffff8018ef0bb78 rsi: 0000000000000000 rdx: 0000000000000000
rcx: fffff8018ef0bb68 r8: fffff8018ef0bb50 r9: fffffe00846ac000
rax: fffff8000175f740 rbx: ffffffff824d8300 rbp: fffffe00846abec0
r10: 0000000000001388 r11: 000000009294bdc3 r12: ffffffff824d8384
r13: fffff8018ef0bb78 r14: 0000000000000000 r15: 0000000000000000
trap number = 12
panic: page fault
cpuid = 4
time = 1726948884
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00846abb10
vpanic() at vpanic+0x131/frame 0xfffffe00846abc40
panic() at panic+0x43/frame 0xfffffe00846abca0
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe00846abd00
trap_pfault() at trap_pfault+0x46/frame 0xfffffe00846abd50
calltrap() at calltrap+0x8/frame 0xfffffe00846abd50
--- trap 0xc, rip = 0xffffffff80bd0d8f, rsp = 0xfffffe00846abe20, rbp = 0xfffffe00846abec0 ---
softclock_call_cc() at softclock_call_cc+0xef/frame 0xfffffe00846abec0
softclock_thread() at softclock_thread+0xe5/frame 0xfffffe00846abef0
fork_exit() at fork_exit+0x7f/frame 0xfffffe00846abf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00846abf30
--- trap 0x44734473, rip = 0xbab2bab2b600b600, rsp = 0x297c297c986e986e, rbp = 0xfcc2fcc225af25af ---
KDB: enter: panic
/var/crash/info.0:
Code: [Select]
Dump header from device: /dev/gpt/swapfs
Architecture: amd64
Architecture Version: 4
Dump Length: 155648
Blocksize: 512
Compression: none
Dumptime: 2024-09-15 18:10:10 -0700
Hostname: OPNSense.localdomain
Magic: FreeBSD Text Dump
Version String: FreeBSD 14.1-RELEASE-p4 stable/24.7-n267825-d0d18dbbaba SMP
Panic String: page fault
Dump Parity: 536253540
Bounds: 0
Dump Status: good
/var/crash/info.1:
Code: [Select]
Dump header from device: /dev/gpt/swapfs
Architecture: amd64
Architecture Version: 4
Dump Length: 155648
Blocksize: 512
Compression: none
Dumptime: 2024-09-17 22:14:54 -0700
Hostname: OPNSense.localdomain
Magic: FreeBSD Text Dump
Version String: FreeBSD 14.1-RELEASE-p4 stable/24.7-n267825-d0d18dbbaba SMP
Panic String: page fault
Dump Parity: 2735905124
Bounds: 1
Dump Status: good
/var/crash/info.2:
Code: [Select]
Dump header from device: /dev/gpt/swapfs
Architecture: amd64
Architecture Version: 4
Dump Length: 155648
Blocksize: 512
Compression: none
Dumptime: 2024-09-21 13:01:24 -0700
Hostname: OPNSense.localdomain
Magic: FreeBSD Text Dump
Version String: FreeBSD 14.1-RELEASE-p4 stable/24.7-n267825-d0d18dbbaba SMP
Panic String: page fault
Dump Parity: 2035653732
Bounds: 2
Dump Status: good
/var/crash/info.3:
Code: [Select]
Dump header from device: /dev/gpt/swapfs
Architecture: amd64
Architecture Version: 4
Dump Length: 155648
Blocksize: 512
Compression: none
Dumptime: 2024-09-23 17:49:30 -0700
Hostname: OPNSense.localdomain
Magic: FreeBSD Text Dump
Version String: FreeBSD 14.1-RELEASE-p4 stable/24.7-n267825-d0d18dbbaba SMP
Panic String: page fault
Dump Parity: 4152328548
Bounds: 3
Dump Status: good
/var/crash/info.4:
Code: [Select]
Dump header from device: /dev/gpt/swapfs
Architecture: amd64
Architecture Version: 4
Dump Length: 155648
Blocksize: 512
Compression: none
Dumptime: 2024-09-24 08:05:57 -0700
Hostname: OPNSense.localdomain
Magic: FreeBSD Text Dump
Version String: FreeBSD 14.1-RELEASE-p4 stable/24.7-n267825-d0d18dbbaba SMP
Panic String: page fault
Dump Parity: 950436196
Bounds: 4
Dump Status: good
/var/crash/textdump.tar.0 can be provided on request