Kernel panics after upgrade to R1

Started by computeralex92, July 16, 2024, 08:21:29 PM

Previous topic - Next topic
Quote from: franco on July 18, 2024, 08:44:27 AM
Thanks, but it's safe to assume the people that matter in this won't appreciate the candidness. Still how does the old saying go? "Do good things and talk about it" is what I'd like to see.

Here's an amended kernel with the proper fix. I also have it on my box so fingers crossed.

# opnsense-update -zkr 24.7.r1_5


Cheers,
Franco

There is as well >
Quote
"Karma is extremely efficient, if one is extremely patient"

Many thanks Franco for taking care of this!

Regards,
S.
Networking is love. You may hate it, but in the end, you always come back to it.

OPNSense HW
APU2D2 - deceased
N5105 - i226-V | Patriot 2x8G 3200 DDR4 | L 790 512G - VM HA(SOON)
N100   - i226-V | Crucial 16G  4800 DDR5 | S 980 500G - PROD

There is room for locking-related issues in pf states handling especially since it's actively being worked on (and I've seen a number of fixes that confirm this). A mildly related change just showed us by allowing a certain path previously not taken to break it, but it could also mean there are more of these issues in other places still. If they manifest only on hardware or due to specific traffic patterns or configuration or plain race conditions between state cleanup kernel thread and active state handling is unclear.


Cheers,
Franco

Probably the ones with Intel Ethernet adapters reported no crashes, I have Realtek, I had installed kernel 24.7.r1_7 and it crashed the moment I started a computer on the LAN side. Maybe it does not like Zenarmor blocking some website.
OPNsense HW:

Minisforum Venus series UN100C, 16 GB RAM, 512 GB SSD
T-bao N9N Pro, 16 GB RAM, 512 GB SSD

I have intel nics and still crashing every few hours with  _5 kernel. (sent crash reports)

Yup, I sent two crash reports, one with _5 and the other _7. Or so I think, since I had bectl-ed beforehand to 24.1 stable before sending the crash reports.
OPNsense HW:

Minisforum Venus series UN100C, 16 GB RAM, 512 GB SSD
T-bao N9N Pro, 16 GB RAM, 512 GB SSD

I haven't seen any crash report with the particular stack trace today matching any of _2, _5 or _7 so far. Also no crash on my main production box.


Cheers,
Franco

Yup, 24.7 did not notice the crash. But bectl-ing to 24.1 and rebooting did see a crash (twice). I don't know if it can see the crash from another bectl.
OPNsense HW:

Minisforum Venus series UN100C, 16 GB RAM, 512 GB SSD
T-bao N9N Pro, 16 GB RAM, 512 GB SSD

Quote from: franco on July 18, 2024, 04:55:45 PM
I haven't seen any crash report with the particular stack trace today matching any of _2, _5 or _7 so far. Also no crash on my main production box.


Cheers,
Franco

I've sent two, one on _5 and one on _7. Have no idea if they made it to you since there is no feedback after sending. I did wait until the wan was up before submitting (since 24.7 the pppoe connection takes a few minutes to come up after reboot)

Edit: just realised you are meaning there is nothing matching this specific crash.

Quote from: almodovaris on July 18, 2024, 05:12:59 PM
Yup, 24.7 did not notice the crash. But bectl-ing to 24.1 and rebooting did see a crash (twice). I don't know if it can see the crash from another bectl.

Not sure about 24.1? We were trying to find the regression between 24.7.b and 24.7.r1 kernel so 24.1.x kernels are very far way from this (FreeBSD 13 vs. 14).


Cheers,
Franco

Quote from: csutcliff on July 18, 2024, 06:19:57 PM
Edit: just realised you are meaning there is nothing matching this specific crash.

Yes, just keep sending if you see one and I'll recheck later. The latest test kernel is

# opnsense-update -zkr 24.7.r1_7

Which may help with two other panics seen before on the 24.7.b kernels.


Cheers,
Franco

Sorry that I was not able to test the kernels today, but now I'm back with kernel 24.7.r1_7...
No panic after reboot; let's see how it is performing.

Regarding the NIC topic:
I'm running on a Intel N100 with Intel I226 NICs.

Quote from: franco on July 18, 2024, 07:22:08 PM
Yes, just keep sending if you see one and I'll recheck later. The latest test kernel is
If 24.1 can see the crash from 24.7, then both crashes are from 24.7. But, again, I don't know if it can report the crashes from another bectl.
OPNsense HW:

Minisforum Venus series UN100C, 16 GB RAM, 512 GB SSD
T-bao N9N Pro, 16 GB RAM, 512 GB SSD

Hmm, ok but that makes searching for these hard because I'm pre-filtering for 24.7 user agent string.

Only noticed r1_7 about 75 minutes ago, applied on the 3 FWs and working fine so far from a crashing perspective

July 18, 2024, 11:07:37 PM #59 Last Edit: July 18, 2024, 11:22:00 PM by almodovaris
Reported by icnl at home dot nl.

The bectl with 24.7 crashed twice. The bectl with 24.1 filled the crash reports. AFAIK 24.1 did not crash, ever. It's a fairly new installation (two days old).

But, okay, it can have misleading data about the installed software.
OPNsense HW:

Minisforum Venus series UN100C, 16 GB RAM, 512 GB SSD
T-bao N9N Pro, 16 GB RAM, 512 GB SSD