Hi All,
I was running OPNSense Business edition fine for amost 2 years but now I'm suffering since about 1 month of repeated kernel panics after random time ( sometimes 4-5 hours, sometimes fater a couple of days ) that reboot one of my DEC2752 or sometimes lock it up and the only solution is to unplug it from power. I cannot figure what is causing these crashes.
I was getting Fatal trap 12 and 9. Then I decided to perform a clean install . Now last time it crashed I got "panic: sleeping thread holds vmobject"
My config:
* ax0: SFP+ DAC cable
* ax1: SFP+ to 10G ethernet adapter ( WAN PPPoe )
* igc0: WAN ( static /29 address )
* gateway switching
* several vlans on ax0
* intervlan routing on vlan4040 as required by ubiquiti switches
* wireguard tunnels to the other 2 DEC2752
* zenarmor
* plugins used: telegraf and wazuh
What i tried so far:
* disable zenarmor, still crashing
* disable plusgins 1 by 1, still crashing
* tried not using ax0 and ax1 interfaces, still crashing
* clean installation with only basic settings and vanilla config ( WAN + PPPoE WAN2 ) still crashing
* opened case and reseated RAM + cleaned every componenet from dust, still crashing
* updated to latest bios, still crashing
* Exported config and loaded it in a proxmox VM does not crash ( but performance sucks )
The same config is working fine on the other 2 DEC2752
I can't find a way to reproduce the crash but this time it seems telegraf related
db:0:kdb.enter.default> bt
Tracing pid 24007 tid 101969 td 0xfffff801cdfd9000
kdb_enter() at kdb_enter+0x33/frame 0xfffffe00be9e1ab0
panic() at panic+0x43/frame 0xfffffe00be9e1b10
propagate_priority() at propagate_priority+0x2af/frame 0xfffffe00be9e1b50
turnstile_wait() at turnstile_wait+0x2e7/frame 0xfffffe00be9e1ba0
__rw_wlock_hard() at __rw_wlock_hard+0x42c/frame 0xfffffe00be9e1c50
vm_object_collapse() at vm_object_collapse+0x861/frame 0xfffffe00be9e1cd0
vmspace_fork() at vmspace_fork+0xaca/frame 0xfffffe00be9e1d30
fork1() at fork1+0x53b/frame 0xfffffe00be9e1da0
sys_fork() at sys_fork+0x54/frame 0xfffffe00be9e1e00
amd64_syscall() at amd64_syscall+0x117/frame 0xfffffe00be9e1f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00be9e1f30
--- syscall (2, FreeBSD ELF64, fork), rip = 0x95719d6, rsp = 0x880127508, rbp = 0x880127620 ---
db:0:kdb.enter.default> ps
pid ppid pgrp uid state wmesg wchan cmd
91109 24007 23981 884 L *vmobjec 0xfffff8017b2a5600 telegraf
90307 24007 23981 884 D vmocol 0xfffff80074039096 telegraf
75377 1 75377 884 Ss select 0xfffff80150327ac0 ping
73525 24007 23981 884 S select 0xfffff80198c4ccc0 ping
13985 1 13985 0 Ss bpf 0xfffff800049cfc00 filterlog
37266 37209 37209 0 S (threaded) python3.11
101259 S kqread 0xfffff80204530d00 python3.11
102157 S uwait 0xfffff80188206700 python3.11
102158 S uwait 0xfffff80191010380 python3.11
102159 S uwait 0xfffff8017bef3680 python3.11
102160 S uwait 0xfffff80188209700 python3.11
37209 1 37209 0 Ss kqread 0xfffff801d2ab5100 daemon
33364 1 33364 59 Ss (threaded) unbound
101443 S kqread 0xfffff80204b87900 unbound
102154 S kqread 0xfffff80204639a00 unbound
102155 S kqread 0xfffff80204a1ea00 unbound
102156 S kqread 0xfffff80204639800 unbound
6039 1 5195 65534 S select 0xfffff801d2405140 dnsmasq
90917 1 90917 0 Ss (threaded) dpinger
101096 S uwait 0xfffff80106a46080 dpinger
102125 S sbwait 0xfffff8004717560c dpinger
102126 S nanslp 0xffffffff81dc5823 dpinger
102127 S accept 0xfffff80047203e10 dpinger
30836 1 30836 0 Ss+ ttyin 0xfffff80001b658b0 getty
1508 1 1508 0 Ss nanslp 0xffffffff81dc5820 cron
97400 32273 97400 0 S select 0xfffff800b5246340 python3.11
96103 32273 96103 0 S select 0xfffff800b5245b40 python3.11
56181 1 55338 0 S (threaded) wazuh-modulesd
101863 S uwait 0xfffff801d26d8a00 wazuh-modulesd
101873 S nanslp 0xffffffff81dc5822 wazuh-modulesd
101874 S select 0xfffff801d26d9c40 wazuh-modulesd
101875 S select 0xfffff801cdfc1e40 wazuh-modulesd
101876 S select 0xfffff801d24fc040 wazuh-modulesd
101877 S uwait 0xfffff801cdfccc00 wazuh-modulesd
101878 S uwait 0xfffff801d26d7600 wazuh-modulesd
101879 S uwait 0xfffff801cdfc1a00 wazuh-modulesd
101880 S uwait 0xfffff801d223e700 wazuh-modulesd
52983 1 52245 0 S (threaded) wazuh-logcollector
100279 S nanslp 0xffffffff81dc5823 wazuh-logcollector
101864 S nanslp 0xffffffff81dc5822 wazuh-logcollector
101866 S nanslp 0xffffffff81dc5823 wazuh-logcollector
101867 S select 0xfffff801d221fe40 wazuh-logcollector
101868 S select 0xfffff801cdfccb40 wazuh-logcollector
101869 S select 0xfffff801cdfc1740 wazuh-logcollector
101870 S select 0xfffff801c4dd6f40 wazuh-logcollector
101871 S select 0xfffff801d221f840 wazuh-logcollector
17911 1 17768 0 S (threaded) wazuh-syscheckd
101131 S nanslp 0xffffffff81dc5820 wazuh-syscheckd
101762 S select 0xfffff801d223e440 wazuh-syscheckd
101763 S uwait 0xfffff801d223e600 wazuh-syscheckd
101764 S nanslp 0xffffffff81dc5823 wazuh-syscheckd
9331 1 9331 0 Ss (threaded) ntpd
101201 S select 0xfffff801d26d85c0 ntpd
94290 1 94148 309 S (threaded) wazuh-agentd
101544 S nanslp 0xffffffff81dc5820 wazuh-agentd
101672 S nanslp 0xffffffff81dc5822 wazuh-agentd
101673 S uwait 0xfffff8018448de00 wazuh-agentd
101674 S nanslp 0xffffffff81dc5821 wazuh-agentd
76461 1 75758 0 S (threaded) wazuh-execd
101283 S uwait 0xfffff80172f24300 wazuh-execd
101636 S select 0xfffff801c4dd62c0 wazuh-execd
41891 1 41447 0 S kqread 0xfffff801975be300 lighttpd
24007 23981 23981 884 L (threaded) telegraf
101571 S uwait 0xfffff800a2e12d00 telegraf
101627 D vm map 0xfffff80037efd798 telegraf
101628 S uwait 0xfffff800a27f5c80 telegraf
101629 S uwait 0xfffff801976e7b80 telegraf
101630 D vm map 0xfffff80037efd798 telegraf
101631 D vm map 0xfffff80037efd798 telegraf
101632 S uwait 0xfffff801976e5280 telegraf
101635 S uwait 0xfffff80197950300 telegraf
101960 S uwait 0xfffff801cdfcc680 telegraf
101961 S wait 0xfffffe00af6505c0 telegraf
101962 S uwait 0xfffff801d26d8600 telegraf
101963 S uwait 0xfffff80172f24700 telegraf
101964 S uwait 0xfffff80004b46680 telegraf
101965 S uwait 0xfffff801cdfc9e00 telegraf
101969 L *vmobjec 0xfffff80001b830c0 telegraf
101970 S uwait 0xfffff8003745a900 telegraf
102182 S uwait 0xfffff800a27f5380 telegraf
105374 S piperd 0xfffffe00b0bf35c0 telegraf
23981 1 23981 0 Ss kqread 0xfffff8015fa56600 daemon
16028 15409 16028 542 Ss select 0xfffff801882090c0 flowd
15409 1 15409 0 Ss sbwait 0xfffff8003787fe0c flowd
92382 1 92382 0 Ss (threaded) dpinger
100364 S uwait 0xfffff80037522c00 dpinger
101568 S sbwait 0xfffff800472afa0c dpinger
101569 S nanslp 0xffffffff81dc5821 dpinger
101570 S accept 0xfffff80037f1a210 dpinger
69099 1 69099 0 Ss nanslp 0xffffffff81dc5821 python3.11
61125 61082 61082 65534 S sbwait 0xfffff80037eefa0c samplicate
61082 1 61082 0 Ss kqread 0xfffff8017eb04e00 daemon
35356 1 35356 0 Ss select 0xfffff8013b2bd640 powerd
85033 84764 84764 0 S nanslp 0xffffffff81dc5821 php
84764 1 84764 0 Ss kqread 0xfffff80197536000 daemon
10753 1581 1581 0 S accept 0xfffff800472ad610 php-cgi
10372 1581 1581 0 S accept 0xfffff800472ad610 php-cgi
10274 1581 1581 0 S accept 0xfffff800472ad610 php-cgi
10012 1581 1581 0 S accept 0xfffff800472ad610 php-cgi
9707 1581 1581 0 S accept 0xfffff800472ad610 php-cgi
9140 708 708 0 S accept 0xfffff80037f52610 php-cgi
8878 2265 2265 0 S accept 0xfffff80037eed210 php-cgi
8215 2265 2265 0 S accept 0xfffff80037eed210 php-cgi
8180 708 708 0 S accept 0xfffff80037f52610 php-cgi
8052 1743 1743 0 S accept 0xfffff800472aa210 php-cgi
7977 2265 2265 0 S accept 0xfffff80037eed210 php-cgi
7338 708 708 0 S accept 0xfffff80037f52610 php-cgi
6793 1743 1743 0 S accept 0xfffff800472aa210 php-cgi
6729 2265 2265 0 S accept 0xfffff80037eed210 php-cgi
6350 1743 1743 0 S accept 0xfffff800472aa210 php-cgi
5425 708 708 0 S accept 0xfffff80037f52610 php-cgi
4714 1743 1743 0 S accept 0xfffff800472aa210 php-cgi
3982 2265 2265 0 S accept 0xfffff80037eed210 php-cgi
3782 708 708 0 S accept 0xfffff80037f52610 php-cgi
3130 1743 1743 0 S accept 0xfffff800472aa210 php-cgi
2265 21 2265 0 Ss wait 0xfffffe008363e040 php-cgi
1743 21 1743 0 Ss wait 0xfffffe00ace9e000 php-cgi
1581 21 1581 0 Ss wait 0xfffffe008363c560 php-cgi
708 21 708 0 Ss wait 0xfffffe00ab578000 php-cgi
21 1 99722 0 S kqread 0xfffff8008b196e00 lighttpd
34598 1 34598 0 Ss select 0xfffff801067686c0 sshd
51490 0 0 0 DL (threaded) [ng_queue]
100464 D sleep 0xffffffff8324e930 [ng_queue0]
100862 D sleep 0xffffffff8324e930 [ng_queue1]
100863 D sleep 0xffffffff8324e930 [ng_queue2]
100864 D sleep 0xffffffff8324e930 [ng_queue3]
51348 1 51348 0 Ss (threaded) mpd5
100867 S select 0xfffff800b5a4e540 mpd5
32273 31517 32273 0 Ss (threaded) syslog-ng
100344 S kqread 0xfffff8008b238700 syslog-ng
101903 S kqread 0xfffff801d218d700 syslog-ng
109615 S kqread 0xfffff80014770000 syslog-ng
109623 S kqread 0xfffff80150244600 syslog-ng
109658 S kqread 0xfffff80054025100 syslog-ng
31517 1 30838 0 S wait 0xfffffe00ac70db00 syslog-ng
872 1 872 0 Ss select 0xfffff8008b7ccc40 devd
499 497 497 0 S (threaded) python3.11
100268 S accept 0xfffff800471caa10 python3.11
497 1 497 0 Ss wait 0xfffffe008363dae0 python3.11
34 0 0 0 DL aiordy 0xfffffe00ab579020 [aiod4]
33 0 0 0 DL aiordy 0xfffffe00836a85c0 [aiod3]
32 0 0 0 DL aiordy 0xfffffe00913fc560 [aiod2]
31 0 0 0 DL aiordy 0xfffffe00913fc000 [aiod1]
19 0 0 0 DL syncer 0xffffffff81dff728 [syncer]
18 0 0 0 DL vlruwt 0xfffffe00913fd020 [vnlru]
17 0 0 0 DL (threaded) [bufdaemon]
100139 D psleep 0xffffffff81dfe728 [bufdaemon]
100140 D - 0xffffffff81a02100 [bufspacedaemon-0]
100141 D - 0xffffffff81a02380 [bufspacedaemon-1]
100142 D - 0xffffffff81a02600 [bufspacedaemon-2]
100143 D - 0xffffffff81a02880 [bufspacedaemon-3]
16 0 0 0 DL psleep 0xffffffff81e33e90 [vmdaemon]
9 0 0 0 DL (threaded) [pagedaemon]
100137 D psleep 0xffffffff81e1c2f8 [dom0]
100144 D launds 0xffffffff81e1c304 [laundry: dom0]
100145 D umarcl 0xffffffff80f7a670 [uma]
8 0 0 0 DL - 0xffffffff81bca8e0 [rand_harvestq]
7 0 0 0 DL pftm 0xffffffff82994920 [pf purge]
6 0 0 0 DL (threaded) [zfskern]
100099 D t->zthr 0xfffff80002039648 [arc_evict]
100100 D t->zthr 0xfffff80002039448 [arc_reap]
100101 D - 0xfffff80002039200 [dbu_evict]
100102 D dbuf_ev 0xffffffff826e6a68 [dbuf_evict_thread]
100103 D - 0xfffff80001a1ec00 [z_vdev_file_0]
100104 D - 0xfffff80001a1ec00 [z_vdev_file_1]
100105 D - 0xfffff80001a1ec00 [z_vdev_file_2]
100106 D - 0xfffff80001a1ec00 [z_vdev_file_3]
100107 D - 0xfffff80001a1ec00 [z_vdev_file_4]
100108 D - 0xfffff80001a1ec00 [z_vdev_file_5]
100109 D - 0xfffff80001a1ec00 [z_vdev_file_6]
100110 D - 0xfffff80001a1ec00 [z_vdev_file_7]
100111 D - 0xfffff80001a1ec00 [z_vdev_file_8]
100112 D - 0xfffff80001a1ec00 [z_vdev_file_9]
100113 D - 0xfffff80001a1ec00 [z_vdev_file_10]
100114 D - 0xfffff80001a1ec00 [z_vdev_file_11]
100115 D - 0xfffff80001a1ec00 [z_vdev_file_12]
100116 D - 0xfffff80001a1ec00 [z_vdev_file_13]
100117 D - 0xfffff80001a1ec00 [z_vdev_file_14]
100118 D - 0xfffff80001a1ec00 [z_vdev_file_15]
100119 D l2arc_f 0xffffffff826e5f68 [l2arc_feed_thread]
100120 D - 0xfffff80002822200 [zfsvfs]
100121 S zevent_ 0xffffffff826e6fc8 [sysevent]
100190 D - 0xfffff80004b91600 [z_zvol]
100191 D - 0xfffff80004b91400 [z_metaslab_0]
100192 D - 0xfffff80004b91400 [z_metaslab_1]
100193 D - 0xfffff80004b62500 [z_prefetch_0]
100194 D - 0xfffff80004b62500 [z_prefetch_1]
100195 D - 0xfffff80004b62500 [z_prefetch_2]
100196 D - 0xfffff80004b62500 [z_prefetch_3]
100197 D - 0xfffff80004b62000 [z_upgrade_0]
100198 D - 0xfffff80004b62000 [z_upgrade_1]
100199 D - 0xfffff80004b62000 [z_upgrade_2]
100200 D - 0xfffff80004b62000 [z_upgrade_3]
100203 D - 0xfffff80004b63600 [dp_sync_taskq_0]
100204 D - 0xfffff80004b63600 [dp_sync_taskq_1]
100205 D - 0xfffff80004b63600 [dp_sync_taskq_2]
100206 D - 0xfffff80004b63300 [dp_zil_clean_taskq_]
100207 D - 0xfffff80004b63300 [dp_zil_clean_taskq_]
100208 D - 0xfffff80004b63300 [dp_zil_clean_taskq_]
100209 D - 0xfffff80004b63300 [dp_zil_clean_taskq_]
100210 D - 0xfffff80004b64800 [z_zrele_0]
100211 D - 0xfffff80004b64800 [z_zrele_1]
100212 D - 0xfffff80004b64800 [z_zrele_2]
100213 D - 0xfffff80004b64800 [z_zrele_3]
100214 D - 0xfffff80004b64300 [z_unlinked_drain_0]
100215 D - 0xfffff80004b64300 [z_unlinked_drain_1]
100216 D - 0xfffff80004b64300 [z_unlinked_drain_2]
100217 D - 0xfffff80004b64300 [z_unlinked_drain_3]
100233 D tx->tx_ 0xfffff800019ecd50 [txg_thread_enter]
100234 D tx->tx_ 0xfffff800019ecd30 [txg_thread_enter]
100235 D mmp->mm 0xfffffe0091391498 [mmp_thread_enter]
100236 D t->zthr 0xfffff80002824748 [z_indirect_condense]
100237 D t->zthr 0xfffff80004b63848 [z_livelist_destroy]
100238 D t->zthr 0xfffff80004b65648 [z_livelist_condense]
100239 D t->zthr 0xfffff80004b65d48 [z_checkpoint_discar]
15 0 0 0 DL (threaded) [usb]
100067 D - 0xfffffe008da2b440 [usbus0]
100068 D - 0xfffffe008da2b498 [usbus0]
100069 D - 0xfffffe008da2b4f0 [usbus0]
100070 D - 0xfffffe008da2b548 [usbus0]
100071 D - 0xfffffe008da2b5a0 [usbus0]
100073 D - 0xfffffe008dbb5440 [usbus1]
100074 D - 0xfffffe008dbb5498 [usbus1]
100075 D - 0xfffffe008dbb54f0 [usbus1]
100076 D - 0xfffffe008dbb5548 [usbus1]
100077 D - 0xfffffe008dbb55a0 [usbus1]
5 0 0 0 DL - 0xffffffff81b58500 [busdma]
4 0 0 0 DL (threaded) [cam]
100057 D - 0xffffffff81b9c240 [doneq0]
100058 D - 0xffffffff81b9c1c0 [async]
100132 D - 0xffffffff81b9c090 [scanner]
3 0 0 0 DL (threaded) [crypto]
100052 D crypto_ 0xffffffff81e197b0 [crypto]
100053 D crypto_ 0xfffff80001697330 [crypto returns 0]
100054 D crypto_ 0xfffff80001697380 [crypto returns 1]
100055 D crypto_ 0xfffff800016973d0 [crypto returns 2]
100056 D crypto_ 0xfffff80001697420 [crypto returns 3]
14 0 0 0 DL seqstat 0xfffff80001825488 [sequencer 00]
13 0 0 0 DL (threaded) [geom]
100044 D - 0xffffffff81d81278 [g_event]
100045 D - 0xffffffff81d81280 [g_up]
100046 D - 0xffffffff81d81288 [g_down]
2 0 0 0 WL (threaded) [clock]
100038 I [clock (0)]
100039 I [clock (1)]
100040 I [clock (2)]
100041 I [clock (3)]
12 0 0 0 WL (threaded) [intr]
100019 I [swi6: task queue]
100020 I [swi6: Giant taskq]
100022 I [swi5: fast taskq]
100042 I [swi1: netisr 0]
100060 I [irq59: nvme0:admin]
100066 I [irq79: xhci0]
100072 I [irq80: xhci1]
100078 I [irq81: hdac0]
100081 I [irq83: ax0]
100082 I [irq84: ax0]
100083 I [irq85: ax0]
100086 I [irq90: ax1]
100087 I [irq91: ax1]
100088 I [irq92: ax1]
100089 I [swi0: uart]
100124 I [swi1: pf send]
100125 I [swi1: pfsync]
100133 I [irq60: nvme0:io0]
100134 I [irq61: nvme0:io1]
100135 I [irq62: nvme0:io2]
100136 I [irq63: nvme0:io3]
11 0 0 0 RL (threaded) [idle]
100003 Run CPU 0 [idle: cpu0]
100004 CanRun [idle: cpu1]
100005 CanRun [idle: cpu2]
100006 Run CPU 3 [idle: cpu3]
1 0 1 0 SLs wait 0xfffffe000ff54040 [init]
10 0 0 0 DL audit_w 0xffffffff81e19ca0 [audit]
0 0 0 0 RLs (threaded) [kernel]
100000 D swapin 0xffffffff81d817f8 [swapper]
100007 D - 0xfffff8000167a500 [softirq_0]
100008 D - 0xfffff8000167a300 [softirq_1]
100009 D - 0xfffff8000167a100 [softirq_2]
100010 D - 0xfffff8000167be00 [softirq_3]
100011 D - 0xfffff8000167bc00 [if_io_tqg_0]
100012 D - 0xfffff8000167ba00 [if_io_tqg_1]
100013 D - 0xfffff8000167b800 [if_io_tqg_2]
100014 D - 0xfffff8000167b600 [if_io_tqg_3]
100015 D - 0xfffff8000167b400 [if_config_tqg_0]
100016 D - 0xfffff8000167b200 [kqueue_ctx taskq]
100017 D - 0xfffff8000167b000 [jail_remove taskq]
100018 D - 0xfffff800011a5d00 [bus taskq]
100021 D - 0xfffff800011a5500 [thread taskq]
100023 D - 0xfffff80001668400 [aiod_kick taskq]
100024 D - 0xfffff80001668200 [deferred_unmount ta]
100025 D - 0xfffff80001668000 [inm_free taskq]
100026 D - 0xfffff80001669d00 [in6m_free taskq]
100027 D - 0xfffff80001669b00 [linuxkpi_irq_wq]
100028 D - 0xfffff80001669900 [linuxkpi_short_wq_0]
100029 D - 0xfffff80001669900 [linuxkpi_short_wq_1]
100030 D - 0xfffff80001669900 [linuxkpi_short_wq_2]
100031 D - 0xfffff80001669900 [linuxkpi_short_wq_3]
100032 D - 0xfffff80001669900 [linuxkpi_short_wq_4]
100033 D - 0xfffff80001669300 [linuxkpi_long_wq_0]
100034 D - 0xfffff80001669300 [linuxkpi_long_wq_1]
100035 D - 0xfffff80001669300 [linuxkpi_long_wq_2]
100036 D - 0xfffff80001669300 [linuxkpi_long_wq_3]
100037 D - 0xfffff80001669300 [linuxkpi_long_wq_4]
100043 D - 0xfffff8000166a400 [firmware taskq]
100048 D - 0xfffff80001678900 [crypto_0]
100049 D - 0xfffff80001678900 [crypto_1]
100050 D - 0xfffff80001678900 [crypto_2]
100051 D - 0xfffff80001678900 [crypto_3]
100061 D - 0xfffff80001788a00 [nvme taskq_0]
100062 D - 0xfffff80001788a00 [nvme taskq_1]
100063 D - 0xfffff80001788500 [if_igc0_tq]
100064 D - 0xfffff80001bbfa00 [if_igc1_tq]
100065 D - 0xfffff80001bc1a00 [if_igc2_tq]
100079 D - 0xfffff80001a1f900 [axgbe dev taskq]
100080 D - 0xfffff80001a1f700 [if_ax0_tq]
100084 D - 0xfffff8000178a500 [axgbe dev taskq]
100085 D - 0xfffff8000178a300 [if_ax1_tq]
100090 D - 0xfffff80001b50c00 [system_taskq_0]
100091 D - 0xfffff80001b50c00 [system_taskq_1]
100092 D - 0xfffff80001b50c00 [system_taskq_2]
100093 D - 0xfffff80001b50c00 [system_taskq_3]
100094 D - 0xfffff80001b50700 [system_delay_taskq_]
100095 D - 0xfffff80001b50700 [system_delay_taskq_]
100096 D - 0xfffff80001b50700 [system_delay_taskq_]
100097 D - 0xfffff80001b50700 [system_delay_taskq_]
100098 D - 0xfffff80002039800 [arc_prune]
100122 D - 0xfffff80002823300 [ip_mroute_tskq task]
100126 D - 0xfffff80002822c00 [acpi_task_0]
100127 D - 0xfffff80002822c00 [acpi_task_1]
100128 D - 0xfffff80002822c00 [acpi_task_2]
100130 D - 0xfffff800011a3600 [mca taskq]
100131 D - 0xfffff80001941b00 [CAM taskq]
100148 D - 0xfffff80004b95000 [z_null_iss]
100149 D - 0xfffff80004b89200 [z_null_int]
100150 D - 0xfffff80004b89000 [z_rd_iss_0]
100151 D - 0xfffff80004b89000 [z_rd_iss_1]
100152 D - 0xfffff80004b89000 [z_rd_iss_2]
100153 D - 0xfffff80004b89000 [z_rd_iss_3]
100154 D - 0xfffff80004b89000 [z_rd_iss_4]
100155 D - 0xfffff80004b89000 [z_rd_iss_5]
100156 D - 0xfffff80004b89000 [z_rd_iss_6]
100157 D - 0xfffff80004b89000 [z_rd_iss_7]
100158 D - 0xfffff80004b8a600 [z_rd_int_0]
100159 D - 0xfffff80004b8a600 [z_rd_int_1]
100160 D - 0xfffff80004b8a600 [z_rd_int_2]
100161 D - 0xfffff80004b8a200 [z_wr_iss_0]
100162 D - 0xfffff80004b8a200 [z_wr_iss_1]
100163 D - 0xfffff80004b8a200 [z_wr_iss_2]
100164 D - 0xfffff80004b8bd00 [z_wr_iss_h_0]
100165 D - 0xfffff80004b8bd00 [z_wr_iss_h_1]
100166 D - 0xfffff80004b8bd00 [z_wr_iss_h_2]
100167 D - 0xfffff80004b8bd00 [z_wr_iss_h_3]
100168 D - 0xfffff80004b8bd00 [z_wr_iss_h_4]
100169 D - 0xfffff80004b8b700 [z_wr_int_0]
100170 D - 0xfffff80004b8b700 [z_wr_int_1]
100171 D - 0xfffff80004b8b700 [z_wr_int_2]
100172 D - 0xfffff80004b8b300 [z_wr_int_h_0]
100173 D - 0xfffff80004b8b300 [z_wr_int_h_1]
100174 D - 0xfffff80004b8b300 [z_wr_int_h_2]
100175 D - 0xfffff80004b8b300 [z_wr_int_h_3]
100176 D - 0xfffff80004b8b300 [z_wr_int_h_4]
100177 D - 0xfffff80004b8cc00 [z_fr_iss_0]
100178 D - 0xfffff80004b8cc00 [z_fr_iss_1]
100179 D - 0xfffff80004b8cc00 [z_fr_iss_2]
100180 D - 0xfffff80004b8c800 [z_fr_int]
100181 D - 0xfffff80004b8c600 [z_cl_iss]
100182 D - 0xfffff80004b8c400 [z_cl_int]
100183 D - 0xfffff80004b8c200 [z_ioctl_iss]
100184 D - 0xfffff80004b8c000 [z_ioctl_int]
100185 D - 0xfffff80004b91d00 [z_trim_iss_0]
100186 D - 0xfffff80004b91d00 [z_trim_iss_1]
100187 D - 0xfffff80004b91d00 [z_trim_iss_2]
100188 D - 0xfffff80004b91d00 [z_trim_iss_3]
100189 D - 0xfffff80004b91800 [z_trim_int]
100909 D - 0xfffff800b5970200 [wg_tqg_0]
100910 RunQ [wg_tqg_1]
100911 RunQ [wg_tqg_2]
100912 D - 0xfffff800b5c23800 [wg_tqg_3]
[19996] Sleeping thread (tid 100343, pid 90307) owns a non-sleepable lock
[19996] KDB: stack backtrace of thread 100343:
[19996] sched_switch() at sched_switch+0x88b/frame 0xfffffe00aac926d0
[19996] mi_switch() at mi_switch+0xbd/frame 0xfffffe00aac926f0
[19996] _blockcount_sleep() at _blockcount_sleep+0x10e/frame 0xfffffe00aac92750
[19996] _vm_page_busy_sleep() at _vm_page_busy_sleep+0x41/frame 0xfffffe00aac92790
[19996] vm_object_collapse_scan_wait() at vm_object_collapse_scan_wait+0x65/frame 0xfffffe00aac927c0
[19996] vm_object_collapse() at vm_object_collapse+0x18c/frame 0xfffffe00aac92840
[19996] vm_object_deallocate() at vm_object_deallocate+0x317/frame 0xfffffe00aac92880
[19996] vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe00aac928a0
[19996] vm_map_remove() at vm_map_remove+0xf9/frame 0xfffffe00aac928d0
[19996] exec_new_vmspace() at exec_new_vmspace+0x245/frame 0xfffffe00aac92930
[19996] exec_elf64_imgact() at exec_elf64_imgact+0x62d/frame 0xfffffe00aac929f0
[19996] kern_execve() at kern_execve+0x7d5/frame 0xfffffe00aac92d80
[19996] sys_execve() at sys_execve+0xb9/frame 0xfffffe00aac92e00
[19996] amd64_syscall() at amd64_syscall+0x117/frame 0xfffffe00aac92f30
[19996] fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00aac92f30
[19996] --- syscall (59, FreeBSD ELF64, execve), rip = 0x95719d6, rsp = 0x880f6c870, rbp = 0x880f6c988 ---
[19996] panic: sleeping thread holds vmobject
[19996] cpuid = 1
[19996] time = 1770442718
[19996] KDB: stack backtrace:
[19996] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00be9e1980
[19996] vpanic() at vpanic+0x161/frame 0xfffffe00be9e1ab0
[19996] panic() at panic+0x43/frame 0xfffffe00be9e1b10
[19996] propagate_priority() at propagate_priority+0x2af/frame 0xfffffe00be9e1b50
[19996] turnstile_wait() at turnstile_wait+0x2e7/frame 0xfffffe00be9e1ba0
[19996] __rw_wlock_hard() at __rw_wlock_hard+0x42c/frame 0xfffffe00be9e1c50
[19996] vm_object_collapse() at vm_object_collapse+0x861/frame 0xfffffe00be9e1cd0
[19996] vmspace_fork() at vmspace_fork+0xaca/frame 0xfffffe00be9e1d30
[19996] fork1() at fork1+0x53b/frame 0xfffffe00be9e1da0
[19996] sys_fork() at sys_fork+0x54/frame 0xfffffe00be9e1e00
[19996] amd64_syscall() at amd64_syscall+0x117/frame 0xfffffe00be9e1f30
[19996] fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00be9e1f30
[19996] --- syscall (2, FreeBSD ELF64, fork), rip = 0x95719d6, rsp = 0x880127508, rbp = 0x880127620 ---
[19996] KDB: enter: panic
Quote from: thomisus on February 07, 2026, 04:31:21 PMWhat i tried so far:
* opened case and reseated RAM + cleaned every componenet from dust, still crashing.
Can these things be booted with a USB Stick that contains MEMTEST86+ somehow ?
Or maybe you can test the RAM in another device ?
I'm running memtest right now. Let's see if it throws any error.
BTW, I got another kernel panic ( same as above ) . At least no more Fatal trap 12 and 9 after clean install
We would recommend a reinstall to be sure the issues go away or keep happening. If they persists please contact sales@deciso.com
Thanks,
Franco