No link with Intel E810 on Opnsense 26.7 / FreeBSD 15 when DDP loaded

[paoba]

I'm not quiet sure if I have the same issue or something related...

I updated to 26.7 and I stopped getting a link on the E823-L. In addition the System behaved very unstable/slow on 26.7 when trying to login directly on the console. It took several seconds just to show the login prompt.

I'm on NVM 3.26 since my NIC is embedded SOC and Supermicro is not providing a newer OEM FW...

[HenrikS]

Same problem with no link on 25GbE, running both E810 and E823 and DDP after upgrading to 26.7
Without loading DDP we get link and about 1Gbps throughput on each NIC :/

E810-XXV fw 7.6.2 api 1.7 nvm 4.60
E823-C fw 7.1.6 api 1.7 nvm 3.26
ICE OS Default Package version 1.3.41.0

[HenrikS]

Upgraded to 26.7.4 as I saw the release notes pointed out that if_ice was updated, no changes, still no link after DDP has loaded.
However I am not sure that the DDP package is?

Has anyone tried to install the newest driver and DDP from Intel 2.1.8?
I have tested to build it on 26.7.4 with the current kernel sources from github, but have not yet tried to install it as I am not quite sure how to perform a roll back and keep future upgrade path clean?

[MajorP93]

Hello,
thanks to everyone who shared their experience in this thread.

We are currently running OPNsense 26.4.1 which is based on FreeBSD 14 and will stay on that version until the issue described in this thread got fixed.

We are using Intel E810-CQDA2 NIC, NVM version 4.80 which seems to work fine under FreeBSD 14. No issues whatsoever.

The issue described in this thread sounds like a regression in FreeBSD 15.

I think it would be great if we had the option to use the official driver from Intel instead of the one integrated in FreeBSD kernel tree.
Maybe that would solve the issue described here.

If OPNSense Team would decide to build and package the driver from intel and offer it as an optional package, it would be greatly appreciated.

Best regards

[Monviech (Cedrik)]

https://forum.opnsense.org/index.php?topic=52881.msg274557#msg274557

[bugacha]

I filed bug with FreeBSD 15.3

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296989

It will take a while until this is fixed tho

Looks like there is some movement on this bug and this can be mainlined into 15 version

[franco]

Added https://github.com/opnsense/src/commit/a5f857190 to stable/26.7 and built a test kernel:

# opnsense-update -zkr 26.7.4-next2

A confirmation would be good because then we can ship it in 26.7.6 next week.


Cheers,
Franco

[paoba]

Added https://github.com/opnsense/src/commit/a5f857190 to stable/26.7 and built a test kernel:

# opnsense-update -zkr 26.7.4-next2

A confirmation would be good because then we can ship it in 26.7.6 next week.


Cheers,
Franco

Hi Franco

Can I upgrade from 26.1.11_10 directly to 26.7.4-next2 using this command? Rollback with Snapshot should be fine right?
If not, I cannot test it because with 26.7 I dont get a link even with DDP Package disabled.

cheers
Patrick

[franco]

Hi Patrick,

Unfortunately the upgrade will want to get the 26.7 kernel. You could try manually booting the 26.7.6 kernel from 26.1.11 which should be fine:

# opnsense-update -fkr 26.7.6 -a FreeBSD:15:amd64/26.7
(reboot)

If it works we can consider bumping the upgrade path from 26.1.11 to 26.7.6. Usually this takes some more time to verify things are still working as expected with the major upgrade also because we have a package manager update in between now.


Cheers,
Franco

[HenrikS]

26.7.6 also seems to have fixed it for me.
I also had to specify dev.ice.x.requested_fec explicit as RS-FEC, as it did not auto-negotiate successfull after DDP load without it set.

[1] ice0: <Intel(R) Ethernet Controller E810-XXV for SFP - 1.43.3-k> mem 0xc2000000-0xc3ffffff,0xc4010000-0xc401ffff at device 0.0 numa-domain 0 on pci6
[1] ice0: Loading the iflib ice driver
[1] ice0: The DDP package was successfully loaded: ICE OS Default Package version 1.3.41.0, track id 0xc0000001.
[1] ice0: fw 7.6.2 api 1.7 nvm 4.60 etid 8001e8d4 netlist 4.3.4000-4.f.0.493a8aab oem 1.3682.0
[1] ice0: Using 4 Tx and Rx queues
[1] ice0: Reserving 4 MSI-X interrupts for iRDMA
[1] ice0: Using MSI-X interrupts with 9 vectors
[1] ice0: Using 1024 TX descriptors and 1024 RX descriptors
[1] ice0: ice_add_rss_cfg on VSI 0 could not configure every requested hash type
[1] ice0: PCI Express Bus: Speed 16.0GT/s Width x8
[1] ice0: pci_iov_attach failed (error=ENOENT)
[1] ice0: link state changed to UP
[1] ice0: Link is up, 25 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[1] ice0: netmap queues/slots: TX 4/1024, RX 4/1024
[1] ice0: link state changed to DOWN
[4] ice0: All configured link modes were attempted but failed to establish link.
[4] ice0: The device will restart the process to establish link.
[4] ice0: Possible Solution: Check link partner connection and configuration.
[25] ice0: Link is up, 25 Gbps Full Duplex, Requested FEC: RS-FEC, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[25] ice0: link state changed to UP

[1] ice2: <Intel(R) Ethernet Connection E823-C for SFP - 1.43.3-k> mem 0xd0000000-0xd7ffffff,0xda010000-0xda01ffff at device 0.0 numa-domain 0 on pci8
[1] ice2: Loading the iflib ice driver
[1] ice2: The DDP package was successfully loaded: ICE OS Default Package version 1.3.41.0, track id 0xc0000001.
[1] ice2: fw 7.1.6 api 1.7 nvm 3.26 etid 8001bc8c netlist 0.0.2768-1.14.1.59461500 oem 1.3429.0
[1] ice2: Using 4 Tx and Rx queues
[1] ice2: Reserving 4 MSI-X interrupts for iRDMA
[1] ice2: Using MSI-X interrupts with 9 vectors
[1] ice2: Using 1024 TX descriptors and 1024 RX descriptors
[1] ice2: ice_add_rss_cfg on VSI 0 could not configure every requested hash type
[1] ice2: link state changed to UP
[1] ice2: Link is up, 25 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[1] ice2: netmap queues/slots: TX 4/1024, RX 4/1024
[5] ice2: All configured link modes were attempted but failed to establish link.
[5] ice2: The device will restart the process to establish link.
[5] ice2: Possible Solution: Check link partner connection and configuration.
[5] ice2: link state changed to DOWN
[25] ice2: Link is up, 25 Gbps Full Duplex, Requested FEC: RS-FEC, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[25] ice2: link state changed to UP

[bugacha]

Added https://github.com/opnsense/src/commit/a5f857190 to stable/26.7 and built a test kernel:

# opnsense-update -zkr 26.7.4-next2

A confirmation would be good because then we can ship it in 26.7.6 next week.


Cheers,
Franco

Thanks a lot Franco, just upgraded to 26.7.6 and everything works fine now out of Opnsense without the need for SRV-IO stuff in Proxmox.

[franco]

neat :D