Changing interface settings on Ralink adapter causes kernel panic since upgrade

[Legally a Shrimp]

2024-08-17 edit: A better workaround has been found by @cercle!
Set the Channel to Auto (in Interfaces -> *WIFI-Interface* -> Common wireless configuration).

2024-07-31 edit: Should anyone face the same issue, there's a workaround!
Set kern.smp.disabled to 1 (in System -> Settings -> Tunables) and reboot.
Applying changes to settings still makes the adapter go down without coming back up until rebooting.



Hi,

apart from the poor, yet for my use-case acceptable performance, I've never had issues using my trusty old Ralink adapter in hostap mode on opnSense until upgrading to 24.7.

Since then my clients can no longer find the WiFi network and applying changes to the settings cause kernel panics.

I understand FreeBSD doesn't play well with WiFi adapters, but I still think this might be some sort of regression and as such should be further investigated.  :(

System information:

Code Select Expand

User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0
FreeBSD 14.1-RELEASE-p2 stable/24.7-n267758-4ad7ad40bc77 SMP amd64
OPNsense 24.7_9 0d38c7804
Plugins os-ddclient-1.22 os-freeradius-1.9.24 os-iperf-1.0_1 os-theme-cicada-1.37
Time Wed, 31 Jul 2024 11:19:50 +0200
OpenSSL 3.0.14
Python 3.11.9
PHP 8.2.20

dmesg.boot:

Code Select Expand

---<<BOOT>>---
Copyright (c) 1992-2023 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.1-RELEASE-p2 stable/24.7-n267758-4ad7ad40bc77 SMP amd64
FreeBSD clang version 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15a9eac9)
VT(vga): resolution 640x480
module run already present!
CPU microcode: updated from 0x7c to 0xf4
CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz (2900.00-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x806e9  Family=0x6  Model=0x8e  Stepping=9
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbb7<SSE3,PCLMULQDQ,DTES64,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x29c67af<FSGSBASE,TSCADJ,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE>
  Structured Extended Features3=0xbc002e00<MCUOPT,MD_CLEAR,TSXFA,IBPB,STIBP,L1DFL,ARCH_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x2000c04<RSBA>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8153300992 (7775 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-119
Launching APs: 1 3 2
random: entropy device external interface
wlan: mac acl policy registered
kbd0 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 15.0.
vtvga0: <VT VGA driver>
smbios0: <System Management BIOS> at iomem 0xf05e0-0xf05fe
smbios0: Version: 2.8, BCD Revision: 2.8
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
acpi0: <ALASKA A M I >
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 24000000 Hz quality 950
Event timer "HPET" frequency 24000000 Hz quality 550
Event timer "HPET1" frequency 24000000 Hz quality 440
Event timer "HPET2" frequency 24000000 Hz quality 440
Event timer "HPET3" frequency 24000000 Hz quality 440
Event timer "HPET4" frequency 24000000 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xde000000-0xdeffffff,0xc0000000-0xcfffffff irq 16 at device 2.0 on pci0
vgapci0: Boot video device
xhci0: <Intel Sunrise Point-LP USB 3.0 controller> mem 0xdf810000-0xdf81ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <serial bus, USB> at device 20.1 (no driver attached)
pci0: <simple comms> at device 22.0 (no driver attached)
ahci0: <Intel Sunrise Point-LP AHCI SATA controller> port 0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xdf828000-0xdf829fff,0xdf82d000-0xdf82d0ff,0xdf82c000-0xdf82c7ff irq 16 at device 23.0 on pci0
ahci0: AHCI v1.31 with 3 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
igb0: <Intel(R) I211 (Copper)> port 0xe000-0xe01f mem 0xdf700000-0xdf71ffff,0xdf720000-0xdf723fff irq 16 at device 0.0 on pci1
igb0: NVM V0.6 imgtype1
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 2 RX queues 2 TX queues
igb0: Using MSI-X interrupts with 3 vectors
igb0: Ethernet address: 40:62:31:06:c5:a7
igb0: netmap queues/slots: TX 2/1024, RX 2/1024
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
pci2: <ACPI PCI bus> on pcib2
igb1: <Intel(R) I211 (Copper)> port 0xd000-0xd01f mem 0xdf600000-0xdf61ffff,0xdf620000-0xdf623fff irq 17 at device 0.0 on pci2
igb1: NVM V0.6 imgtype1
igb1: Using 1024 TX descriptors and 1024 RX descriptors
igb1: Using 2 RX queues 2 TX queues
igb1: Using MSI-X interrupts with 3 vectors
igb1: Ethernet address: 40:62:31:06:c5:a8
igb1: netmap queues/slots: TX 2/1024, RX 2/1024
pcib3: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci3: <ACPI PCI bus> on pcib3
igb2: <Intel(R) I211 (Copper)> port 0xc000-0xc01f mem 0xdf500000-0xdf51ffff,0xdf520000-0xdf523fff irq 18 at device 0.0 on pci3
igb2: NVM V0.6 imgtype1
igb2: Using 1024 TX descriptors and 1024 RX descriptors
igb2: Using 2 RX queues 2 TX queues
igb2: Using MSI-X interrupts with 3 vectors
igb2: Ethernet address: 40:62:31:06:c5:a9
igb2: netmap queues/slots: TX 2/1024, RX 2/1024
pcib4: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci4: <ACPI PCI bus> on pcib4
igb3: <Intel(R) I211 (Copper)> port 0xb000-0xb01f mem 0xdf400000-0xdf41ffff,0xdf420000-0xdf423fff irq 19 at device 0.0 on pci4
igb3: NVM V0.6 imgtype1
igb3: Using 1024 TX descriptors and 1024 RX descriptors
igb3: Using 2 RX queues 2 TX queues
igb3: Using MSI-X interrupts with 3 vectors
igb3: Ethernet address: 40:62:31:06:c5:aa
igb3: netmap queues/slots: TX 2/1024, RX 2/1024
pcib5: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci5: <ACPI PCI bus> on pcib5
igb4: <Intel(R) I211 (Copper)> port 0xa000-0xa01f mem 0xdf300000-0xdf31ffff,0xdf320000-0xdf323fff irq 16 at device 0.0 on pci5
igb4: NVM V0.6 imgtype1
igb4: Using 1024 TX descriptors and 1024 RX descriptors
igb4: Using 2 RX queues 2 TX queues
igb4: Using MSI-X interrupts with 3 vectors
igb4: Ethernet address: 40:62:31:06:c5:ab
igb4: netmap queues/slots: TX 2/1024, RX 2/1024
pcib6: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
pci6: <ACPI PCI bus> on pcib6
igb5: <Intel(R) I211 (Copper)> port 0x9000-0x901f mem 0xdf200000-0xdf21ffff,0xdf220000-0xdf223fff irq 17 at device 0.0 on pci6
igb5: NVM V0.6 imgtype1
igb5: Using 1024 TX descriptors and 1024 RX descriptors
igb5: Using 2 RX queues 2 TX queues
igb5: Using MSI-X interrupts with 3 vectors
igb5: Ethernet address: 40:62:31:06:c5:ac
igb5: netmap queues/slots: TX 2/1024, RX 2/1024
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory> at device 31.2 (no driver attached)
hdac0: <Intel Kaby Lake-LP HDA Controller> mem 0xdf820000-0xdf823fff,0xdf800000-0xdf80ffff irq 16 at device 31.3 on pci0
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
ns8250: UART FCR is broken
ns8250: UART FCR is broken
uart0: <16950 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
ns8250: UART FCR is broken
uart0: console (115200,n,8,1)
ns8250: UART FCR is broken
ns8250: UART FCR is broken
uart1: <16950 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xcffff pnpid ORM0000 on isa0
hwpstate_intel0: <Intel Speed Shift> on cpu0
hwpstate_intel1: <Intel Speed Shift> on cpu1
hwpstate_intel2: <Intel Speed Shift> on cpu2
hwpstate_intel3: <Intel Speed Shift> on cpu3
Timecounter "TSC-low" frequency 1451999960 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
hdacc0: <Intel Kaby Lake HDA CODEC> at cad 2 on hdac0
hdaa0: <Intel Kaby Lake Audio Function Group> at nid 1 on hdacc0
pcm0: <Intel Kaby Lake (HDMI/DP 8ch)> at nid 3 on hdaa0
Trying to mount root from zfs:zroot/ROOT/default []...
ugen0.1: <Intel XHCI root HUB> at usbus0
uhub0 on usbus0
uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
ada0: <Samsung SSD 860 EVO mSATA 250GB RVT41B6Q> ACS-4 ATA SATA 3.x device
ada0: Serial Number S41MNC0KA10520Y
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 238475MB (488397168 512 byte sectors)
ada0: quirks=0x3<4K,NCQ_TRIM_BROKEN>
uhub0: 18 ports with 18 removable, self powered
Root mount waiting for: usbus0
ugen0.2: <Ralink 802.11 n WLAN> at usbus0
run0 on uhub0
run0: <Ralink 802.11 n WLAN, class 0/0, rev 2.00/1.01, addr 1> on usbus0
run0: MAC/BBP RT3070 (rev 0x0201), RF RT3020 (MIMO 1T1R), address 00:08:ca:51:cd:e3
run0: [HT] Enabling 802.11n

info.0:

Code Select Expand

Dump header from device: /dev/ada0p3
  Architecture: amd64
  Architecture Version: 4
  Dump Length: 76288
  Blocksize: 512
  Compression: none
  Dumptime: 2024-07-31 11:17:56 +0200
  Hostname: opnsense.home.arpa
  Magic: FreeBSD Text Dump
  Version String: FreeBSD 14.1-RELEASE-p2 stable/24.7-n267758-4ad7ad40bc77 SMP
  Panic String: page fault
  Dump Parity: 1831400993
  Bounds: 0
  Dump Status: good

textdump attached

[franco]

Tracing pid 0 tid 100123 td 0xfffff80009ad3740
kdb_enter() at kdb_enter+0x33/frame 0xfffffe0084fd8b90
panic() at panic+0x43/frame 0xfffffe0084fd8bf0
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe0084fd8c50
trap_pfault() at trap_pfault+0x46/frame 0xfffffe0084fd8ca0
calltrap() at calltrap+0x8/frame 0xfffffe0084fd8ca0
--- trap 0xc, rip = 0xffffffff80d5403f, rsp = 0xfffffe0084fd8d70, rbp = 0xfffffe0084fd8dd0 ---
ieee80211_beacon_update() at ieee80211_beacon_update+0x32f/frame 0xfffffe0084fd8dd0
run_update_beacon() at run_update_beacon+0x152/frame 0xfffffe0084fd8e10
vap_update_wme() at vap_update_wme+0x87/frame 0xfffffe0084fd8e40
taskqueue_run_locked() at taskqueue_run_locked+0x182/frame 0xfffffe0084fd8ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe0084fd8ef0
fork_exit() at fork_exit+0x7f/frame 0xfffffe0084fd8f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0084fd8f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Hmm, this isn't the first stack trace ending in ieee80211_beacon_update() I've seen... ath driver has the same issue I think.

I'll put it on my list of things to look at.


Cheers,
Franco

[franco]

> Set kern.smp.disabled to 1 (in System -> Settings -> Tunables) and reboot.

I don't know how far this goes but knowing that FreeBSD wants to get rid of the "Giant" lock and going back to disable SMP to fix random things would mean that the lock was removed where it was still needed and/or not properly replaced?


Cheers,
Franco

[franco]

Ref: https://forum.opnsense.org/index.php?topic=41816.0

[Legally a Shrimp]

I'll put it on my list of things to look at.

That's all I could ask for, thanks! :D

[franco]

Ref: https://forum.opnsense.org/index.php?topic=41891.0

[jhelmly]

Unfortunately, neither of the settings mentioned seem to have no bearing on my situation.  Just upgraded from 24.1.10 to 24.7.2 (two upgrades), and now our WIFI is not working.  The channels was already set to auto.  After adding the kern.smb.disabled==1 setting, I am still only getting frequent "hostapd 51964 - [meta sequenceId="1"] ath0_wlan1: WPA rekeying GTK" posted to the wireless_20240823.log.  I am likely going to need to downgrade back to 24.1.10 to get our WIFI back up as people need it.

[guttermonk]

@jhelmly I'm in the same boat. I've started a new issue: https://github.com/opnsense/core/issues/8080