Intel x710 and Multicast traffic

[furfix]

Hi All, hope somebody can help me a little with this.

I have a box where I'm running OPNSense baremetal that comes with i226 nic and x710 nic. Since my ISP handoff is fiber, I'm using the x710 one, but after long hours, I came to the conclusion that Multicast traffic is not working with the x710. Same config, but using the i226 for the LAN works fine. I've also installed a Mellanox card on the box to be sure that the Intel x710 is the problem, and indeed...with the Mellanox Multicast is working fine.

I would really appreciate if somebody can help me to narrow the issue, because I don't know what else to do to keep troubleshooting the problem.

Is it maybe possible to update the Intel Drivers? I read it is not recommended to install drivers to avoid issues with OPNsense.

Code Select Expand

<5>ixl0: Link is up, 1 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: None, Autoneg: True, Flow Control: None
<6>ixl0: link state changed to UP

<5>ixl1: Link is up, 1 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: None, Autoneg: True, Flow Control: None
<6>ixl1: link state changed to UP


I'm not sure if it has something to do with this, but also i see a LOT of these msgs:

Code Select Expand

<6>ixl0: Disabled multicast promiscuous mode
<6>ixl1: Disabled multicast promiscuous mode

I've updated also the Firmware of the x710, but no changes:

Code Select Expand

root@opnsense:~ # sysctl dev.ixl.0.fw_version ; sysctl dev.ixl.0.iflib.driver_version
dev.ixl.0.fw_version: fw 9.150.77492 api 1.15 nvm 9.50 etid 8000f12c oem 0.0.0
dev.ixl.0.iflib.driver_version: 2.3.3-k


Code Select Expand

root@opnsense:~ # ifconfig -vv ixl0
ixl0: flags=1008a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: ONT (opt11)
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether xx:xx:xx:xx:xx:xx
        inet6 xx::xx:xx:xx:xx%ixl0 prefixlen 64 scopeid 0x1
        groups: Interfaces
        media: Ethernet autoselect (1000baseLX <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        drivername: ixl0
        plugged: SFP/SFP+/SFP28 1000BASE-LX (SC)
        vendor: FS PN: SFP-GE-BX SN: C2401543867 DATE: 2024-01-30
        module temperature: 46.89 C voltage: 3.27 Volts
        lane 1: RX power: 0.21 mW (-6.82 dBm) TX bias: 21.84 mA


Code Select Expand

root@opnsense:~ # ifconfig -vv ixl1
ixl1: flags=1008a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: LAN (lan)
        options=4900028<VLAN_MTU,JUMBO_MTU,NETMAP,HWSTATS,MEXTPG>
        ether XX:XX:XX:XX:XX:XX
        inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
        inet6 xxxx::xxxx:xxxx:xxxx:xxxx%ixl1 prefixlen 64 scopeid 0x2
        inet6 xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxxprefixlen 64
        groups: Interfaces
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        drivername: ixl1
        plugged: SFP/SFP+/SFP28 1000BASE-T (RJ45)
        vendor: Ubiquiti Inc. PN: CM-RJ45-1G SN: XXXXXXXXXXX DATE: 2024-01-03


Code Select Expand

ixl0@pci0:2:0:0:        class=0x020000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1572 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller X710 for 10GbE SFP+'
    class      = network
    subclass   = ethernet
ixl1@pci0:2:0:1:        class=0x020000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1572 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller X710 for 10GbE SFP+'
    class      = network
    subclass   = ethernet


On OPNSense Logs:

Code Select Expand

2024-08-02T09:35:28 Warning igmpproxy select() failure; Errno(4): Interrupted system call
2024-08-02T09:35:28 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (execute task : igmpproxy_configure_do())
2024-08-02T09:35:27 Warning igmpproxy select() failure; Errno(4): Interrupted system call
2024-08-02T09:35:27 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (execute task : igmpproxy_configure_do())
2024-08-02T09:35:27 Warning igmpproxy select() failure; Errno(4): Interrupted system call
2024-08-02T09:35:27 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (execute task : igmpproxy_configure_do())

Any idea, will be much appreciated!

[newsense]

Check if this kernel fixes it for you.

Code Select Expand

opnsense-update -zkr 24.7_11 && opnsense-shell reboot

[franco]

"_11" was a bit of a hit and miss and I have removed it now. I can recommend "_10" as that is definitely a patch set that is part of 24.7.1 and has IGMP specific fixes:

# opnsense-update -zkr 24.7_10 && opnsense-shell reboot


Cheers,
Franco

[furfix]

Check if this kernel fixes it for you.

Code Select Expand

opnsense-update -zkr 24.7_11 && opnsense-shell reboot

Hi! Just tried, but I get this error:

Code Select Expand

Fetching kernel-24.7_11-amd64.txz: ..[fetch: https://pkg.opnsense.org/FreeBSD:14:amd64/snapshots/sets/kernel-24.7_11-amd64.txz.sig: Not Found] failed, no signature found


[franco]

Why, yes. ;)

[furfix]

Sorry, replied at the same time. Ive tried 24.7_10 yesterday, but no luck. The issue remains in my case (and for the records, just with the x710 NIC). Both i226 and Mellanox were working fine, even with the current 24.7 kernel, but somehow the x710 doesn't like it.

[furfix]

Why, yes. ;)

I tried again the _10 for the sake of trying to solve it, but still IGMP/Multicast seems to no be working for this card. If you think I can troubleshoot it somehow or share more logs, or anything you  may need, just let me know. :)

[furfix]

@franco, is there anything I can do to help? Should I open an issue in Github with this info? What other information would be helpful?

Thanks in advance!!

[franco]

Did this start with 24.7 or was this always going on? I'm asking to see if I should check driver commits or dig somewhere else.


Cheers,
Franco

[franco]

GitHub link https://github.com/opnsense/src/issues/212

[franco]

Working test kernel :) https://github.com/opnsense/src/issues/212#issuecomment-2274126169

[furfix]

@franco I've sent you via DM the outcome of the commands Intel asked to run. If I can help with anything else, please do not hesitate to let me know.

[franco]

I'm currently too busy until 24.7.2 is out, but I will try to follow up on it later this week. Sorry.


Cheers,
Franco