DEC840 gets stuck on boot if ax1 is linked

[feld]

OPNsense 26.1.8_5-amd64

I have Quantum Fiber for my internet service and my CPE equipment is a C5500XK. Unfortunately I don't have a GPON SFP that will let me bypass this equipment at this time.

I have a 10GBASE-T SFP+ 30m https://www.fs.com/products/111919.html

If I attempt to boot my DEC840 with this physical link up it gets stuck and fails to complete booting. I have tried replacing this SFP as I have many on hand, but they all produce the same behavior. Changing this out for an 1G transceiver works fine and it boots noticeably faster than the 10G.

I am attaching the boot log captured over serial console at boot via script+screen. I had to clean this up a bit as it had all the terminal control characters and color sequences littered throughout but I think it's fully intact.

What I have observed is that there are some additional i2c devices detected when the 10G module is used, and it seems to fail to configure/communicate with the SFP based on the timeouts it displays in the boot log.

If I boot the firewall without the physical link up (unplug ethernet), and then plug in after fully booted it works fine -- or at least I don't notice any issues with the link or its stability.


edit: I want to mention that this hardware configuration worked fine for a couple years, but I can't be certain when this problem started happening. I believe it started somewhere within the 25.x series.

[tuto2]

Hi,

I took the liberty of testing with an older DEC850 model (same platform, 8 instead of 4 cores). Since I had both the FS 10GSR-85 and the FS 10G-T lying around, plugged these in as well but can't seem to reproduce your issue.

It can be helpful to set both dev.ax.0.axgbe_debug_level and dev.ax.1.axgbe_debug_level to "1" in system -> settings -> tunables, and reboot. This will produce a lot of noise over serial so make sure you can access via ssh to reset these tunables by hand (# sysctl dev.ax.0|1.axgbe_debug_level=0) or make sure you're able to access the GUI to reset the tunables there.

Instead of directly fetching from the serial console you can also pipe "# dmesg" into a file, but make sure it has the ---<<BOOT>>--- in there.

Possibly unrelated, but is one of the ax ports configured for DHCP as a client?

[feld]

Instead of directly fetching from the serial console you can also pipe "# dmesg" into a file, but make sure it has the ---<<BOOT>>--- in there.

I can't, because it doesn't finish booting to the point where I can get a shell and look at dmesg :)

> Possibly unrelated, but is one of the ax ports configured for DHCP as a client?

that is correct, ax1 is a DHCP client to my ISP