Mellanox ConnectX-4 Lx (MT27710) on new installation 25.1.3-amd64

[talbaror]

**Hello,** 

I'm in the process of migrating from **pfSense to OPNsense** and have installed OPNsense on the **same hardware** as my current running pfSense setup. 

The server includes: 
- **Mellanox ConnectX-4 Lx (MT27710)** 
- **Intel x740 4x10G** interfaces 
- **Intel 4x1Gbit** interfaces 

On pfSense, all interfaces were **automatically detected**, but on OPNsense, they are **not appearing in Interfaces: Assignments**. 

I checked the drivers, and they **seem to be loaded** when running `kldstat`, but the interfaces are still missing. 
I also tried the following commands to load the drivers: 

Code Select Expand

echo 'mlx5en_load="YES"' >> /boot/loader.conf
echo 'if_igb_load="YES"' >> /boot/loader.conf
However, these settings **disappear after reboot**. I also attempted to add them under **System > Settings > Tunables**, but it didn't resolve the issue. 

Has anyone encountered this before? Any suggestions on how to make OPNsense recognize these interfaces properly? 

Thanks in advance!
 

Code Select Expand

pciconf -lv | grep -A4 -i 'network\|ethernet'
    device     = '82576 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb1@pci0:20:0:1:       class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1526 subvendor=0x8086 subdevice=0xa06c
    vendor     = 'Intel Corporation'
    device     = '82576 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb2@pci0:21:0:0:       class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1526 subvendor=0x8086 subdevice=0xa06c
    vendor     = 'Intel Corporation'
    device     = '82576 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb3@pci0:21:0:1:       class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1526 subvendor=0x8086 subdevice=0xa06c
    vendor     = 'Intel Corporation'
    device     = '82576 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
pcib9@pci0:54:0:0:      class=0x060400 rev=0x07 hdr=0x01 vendor=0x8086 device=0x2030 subvendor=0x1590 subdevice=0x00ea
    vendor     = 'Intel Corporation'
    device     = 'Sky Lake-E PCI Express Root Port A'
    class      = bridge
--
    class      = network
    subclass   = ethernet
mlx5_core1@pci0:93:0:1: class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1015 subvendor=0x1590 subdevice=0x00d3
    vendor     = 'Mellanox Technologies'
    device     = 'MT27710 Family [ConnectX-4 Lx]'
    class      = network
    subclass   = ethernet
none114@pci0:128:4:0:   class=0x088000 rev=0x07 hdr=0x00 vendor=0x8086 device=0x2021 subvendor=0x1590 subdevice=0x00ea
    vendor     = 'Intel Corporation'
    device     = 'Sky Lake-E CBDMA Registers'
    class      = base peripheral
--
    class      = network
    subclass   = ethernet


[Patrick M. Hausen]

The method via the UI is the recommended one. If the modules show up in "kldstat" after a reboot that part is well.

Are you running the latest version 25.1?

Do the interfaces show up being detected in "cat /var/run/dmesg.boot"?

Do the interfaces show up in "ifconfig -a"?

[talbaror]

Hi
Thanks for the answer
version: 25.1.3-amd64
"cat /var/run/dmesg.boot" show

Code Select Expand

mlx5_core: INFO: (mlx5_core0): E-Switch: Total vports 9, l2 table size(65536), per vport: max uc(1024) max mc(16384)
mlx5_core1: <mlx5_core> mem 0xe0000000-0xe1ffffff at device 0.1 numa-domain 0 on pci13
ifconfig -a , shows only actual interfaces that available to assign
Thank you

Code Select Expand

ifconfig -a
igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: WAN (wan)
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether 90:e2:ba:10:b3:b8
        inet 212.143.237.130 netmask 0xffffff00 broadcast 212.143.237.255
        inet6 fe80::92e2:baff:fe10:b3b8%igb0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
igb1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether 90:e2:ba:10:b3:b9
        media: Ethernet autoselect
        status: no carrier
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
igb2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether 90:e2:ba:10:b3:ba
        media: Ethernet autoselect
        status: no carrier
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
igb3: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: LAN (lan)
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether 90:e2:ba:10:b3:bb
        inet 212.143.237.6 netmask 0xffffff00 broadcast 212.143.237.255
        inet6 fe80::92e2:baff:fe10:b3bb%igb3 prefixlen 64 scopeid 0x4
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0 metric 0 mtu 1536
        options=0
        groups: enc
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33152
        options=0
        groups: pflog
pfsync0: flags=0 metric 0 mtu 1500
        options=0
        maxupd: 128 defer: off version: 1400
        syncok: 1
        groups: pfsync
mce0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=7e8a00a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,NV,LINKSTATE,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6>
        ether 94:40:c9:94:76:20
        media: Ethernet autoselect <full-duplex,rxpause,txpause>
        status: no carrier (Cable is unplugged.)
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
mce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=7e8a00a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,NV,LINKSTATE,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6>
        ether 94:40:c9:94:76:21
        media: Ethernet autoselect <full-duplex,rxpause,txpause>
        status: no carrier (Cable is unplugged.)
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>


[patient0]

ifconfig -a , shows only actual interfaces that available to assign

You got 4 igbX interfaces, these are the 4 1Gbit and 2 Mellanox ConnectX-4 Lx interfaces, the mce0 & mce1.

I can't find infos about Intel x740, can you link to a product page?

[Patrick M. Hausen]

So what's the problem? mce0 and mce1 are your Mellanox based interfaces:

https://man.freebsd.org/cgi/man.cgi?query=mlx5en&sektion=4&format=html


For the 10G Intel ports try "if_ixl_load" instead of "if_igb_load":

https://man.freebsd.org/cgi/man.cgi?query=ixl&sektion=4&format=html


"if_igb_load" should not be necessary and the igb[0123] ports should show up automatically. These are your 1G ports.


EDIT: should the ixl driver fail to detect your Intel 10G interfaces you can try the vendor driver by installing from ports:

https://www.freshports.org/net/intel-ixl-kmod/

[talbaror]

ohh sorry, i thought that mce0,mce1 are the HP 10G daughter interface (its hp dl360 gen10)

as for the Intel , i will give it a try
Thanks a lot