No link with Intel E810 on Opnsense 26.7 / FreeBSD 15 when DDP loaded

[paoba]

thanks for the heads up! here some more details about my interfaces and my switch. I was supprised my switch thinks the opnsense LAN is connected by Medium-type "Fiber", which is incorrect.

I don't know what FEC settings my internet provider has on their switch, but if required I can open up a ticket to find out.

I will try the tunable for both ports and see what happens after a reboot and also try to upgrade again.

testing tunables:

Code Select Expand

dev.ice.0.requested_fec="RS-FEC"
dev.ice.1.requested_fec="RS-FEC"

Switch opnsense LAN Port details:

Code Select Expand

Medium-type is Fiber
Admin duplex mode is AUTO, oper duplex is Full
Admin speed is 25G, oper speed is 25G
Flow control admin status is OFF, flow control oper status is OFF
Admin negotiation mode is OFF, oper negotiation state is OFF
Admin FEC mode is auto, oper FEC mode is rs

LAN Interface (ice0):

Code Select Expand

dev.ice.0.requested_fec: RS-FEC
dev.ice.0.negotiated_fec: RS-FEC
dev.ice.0.current_speed: 25 Gbps
media: Ethernet autoselect (25G-AUI <full-duplex>)
plugged: SFP/SFP+/SFP28 25GBASE-CR CA-25G-S (Copper pigtail)
WAN Interface (ice1):

Code Select Expand

dev.ice.1.requested_fec: RS-FEC
dev.ice.1.negotiated_fec: RS-FEC
dev.ice.1.current_speed: 25 Gbps
media: Ethernet autoselect (25GBase-LR <full-duplex>)
plugged: SFP/SFP+/SFP28 100GBASE-LR4 or 25GBASE-LR (LC)

[paoba]

well that worked better than expected... after I set both tunables, I could upgrade without issues!

thanks alot for the help! love this community!

So I guess it's better to keep the tunables in the future? is there anything you need from my side?

here is the dmesg log from 27.7.7:

Code Select Expand

[1] ice0: <Intel(R) Ethernet Connection E823-L for SFP - 1.43.3-k> mem 0xf0000000-0xf7ffffff,0xfa010000-0xfa01ffff at device 0.0 numa-domain 0 on pci10
[1] ice0: Loading the iflib ice driver
[1] ice0: The DDP package was successfully loaded: ICE OS Default Package version 1.3.41.0, track id 0xc0000001.
[1] ice0: fw 7.2.14 api 1.7 nvm 3.42 etid 8001ff0f netlist 0.1.7000-1.25.0.f083a9d5 oem 1.3755.0
[1] ice0: Using 10 Tx and Rx queues
[1] ice0: Reserving 10 MSI-X interrupts for iRDMA
[1] ice0: Using MSI-X interrupts with 21 vectors
[1] ice0: Using 1024 TX descriptors and 1024 RX descriptors
[1] ice0: Ethernet address: ██:██:██:██:██:██
[1] ice0: ice_add_rss_cfg on VSI 0 could not configure every requested hash type
[1] ice0: Firmware LLDP agent disabled
[1] ice0: link state changed to UP
[1] ice0: Link is up, 25 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[1] ice0: netmap queues/slots: TX 10/1024, RX 10/1024
[1] ice0: link state changed to DOWN
[9] ice0: All configured link modes were attempted but failed to establish link.
[9] ice0: The device will restart the process to establish link.
[9] ice0: Possible Solution: Check link partner connection and configuration.
[47] ice0: Link is up, 25 Gbps Full Duplex, Requested FEC: RS-FEC, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[47] ice0: link state changed to UP
[48] ice0: Failed to add VLAN filters:
[48] ice0: - vlan 100, status -14
[48] ice0: Failure adding VLAN 100 to main VSI, err ICE_ERR_ALREADY_EXISTS aq_err OK


Code Select Expand

[1] ice1: <Intel(R) Ethernet Connection E823-L for SFP - 1.43.3-k> mem 0xe8000000-0xefffffff,0xfa000000-0xfa00ffff at device 0.2 numa-domain 0 on pci10
[1] ice1: Loading the iflib ice driver
[1] ice1: DDP package already present on device: ICE OS Default Package version 1.3.41.0, track id 0xc0000001.
[1] ice1: fw 7.2.14 api 1.7 nvm 3.42 etid 8001ff0f netlist 0.1.7000-1.25.0.f083a9d5 oem 1.3755.0
[1] ice1: Using 10 Tx and Rx queues
[1] ice1: Reserving 10 MSI-X interrupts for iRDMA
[1] ice1: Using MSI-X interrupts with 21 vectors
[1] ice1: Using 1024 TX descriptors and 1024 RX descriptors
[1] ice1: Ethernet address: ██:██:██:██:██:██
[1] ice1: ice_add_rss_cfg on VSI 0 could not configure every requested hash type
[1] ice1: Firmware LLDP agent disabled
[1] ice1: link state changed to UP
[1] ice1: Link is up, 25 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[1] ice1: netmap queues/slots: TX 10/1024, RX 10/1024
[6] ice1: All configured link modes were attempted but failed to establish link.
[6] ice1: The device will restart the process to establish link.
[6] ice1: Possible Solution: Check link partner connection and configuration.
[6] ice1: link state changed to DOWN
[46] ice1: Link is up, 25 Gbps Full Duplex, Requested FEC: RS-FEC, Negotiated FEC: RS-FEC, Autoneg: False, Flow Control: None
[46] ice1: link state changed to UP


[franco]

Nope, that's great news.

I've worked today on enabling the upgrade path to 26.1.11 -> 26.7.7. It looks good from my perspective so expecting this to be live later today.


Cheers,
Franco

[bugacha]

well that worked better than expected... after I set both tunables, I could upgrade without issues!

thanks alot for the help! love this community!

So I guess it's better to keep the tunables in the future? is there anything you need from my side?

Yes just keep them.

Something in BSD ICE driver doesn't properly negotiate FEC when network switch is in Auto mode.
I've always had it on E810 card.

Linux ICE driver doesn't have this issue.