Topics

I am trying to enable a 10GbE card based on QLogic 41000 series.
I am using OPNsense 22.7.4 (amd64/OpenSSL).

To do this, I have to enable a driver:

Code Select Expand


kldload if_qlnxe


The card seems to be detected correctly:

Code Select Expand


ql0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO>
ether 34:80:0d:04:fc:2c
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ql1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO>
ether 34:80:0d:04:fc:2d
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ql2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO>
ether 34:80:0d:04:fc:2e
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ql3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO>
ether 34:80:0d:04:fc:2f
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


dmesg, however,  shows something is not going great

Code Select Expand


intsmb0: <AMD FCH SMBus Controller> port 0xff00-0xff0f at device 20.0 on pci0
intsmb0: Could not allocate I/O space
device_attach: intsmb0 attach returned 6
ql0: <Qlogic 10GbE/25GbE/40GbE PCI CNA (AH) Adapter-Ethernet Function v2.0.112> mem 0xea860000-0xea87ffff,0xea000000-0xea7fffff,0xea8b0000-0xea8bffff irq 50 at device 0.0 on pci11
ql0: qlnx_set_personality: ETH_IWARP
ql0: setting parameters required by iWARP dev
ql0: Ethernet address: 34:80:0d:04:fc:2c
ql1: <Qlogic 10GbE/25GbE/40GbE PCI CNA (AH) Adapter-Ethernet Function v2.0.112> mem 0xea840000-0xea85ffff,0xe9800000-0xe9ffffff,0xea8a0000-0xea8affff irq 50 at device 0.1 on pci11
ql1: qlnx_set_personality: ETH_IWARP
ql1: setting parameters required by iWARP dev
ql1: Ethernet address: 34:80:0d:04:fc:2d
ql2: <Qlogic 10GbE/25GbE/40GbE PCI CNA (AH) Adapter-Ethernet Function v2.0.112> mem 0xea820000-0xea83ffff,0xe9000000-0xe97fffff,0xea890000-0xea89ffff irq 50 at device 0.2 on pci11
ql2: qlnx_set_personality: ETH_IWARP
ql2: setting parameters required by iWARP dev
ql2: Ethernet address: 34:80:0d:04:fc:2e
ql3: <Qlogic 10GbE/25GbE/40GbE PCI CNA (AH) Adapter-Ethernet Function v2.0.112> mem 0xea800000-0xea81ffff,0xe8800000-0xe8ffffff,0xea880000-0xea88ffff irq 50 at device 0.3 on pci11
ql3: qlnx_set_personality: ETH_IWARP
ql3: setting parameters required by iWARP dev
ql3: Ethernet address: 34:80:0d:04:fc:2f


But the main problem is that I I get a LOT of ifconfig instances consuming all the CPU.
And the admin website hangs and the machine needs to be rebooted.
Of course, if I try to enable this at bootime in loader.conf, the server does not complete booting.

Code Select Expand


last pid: 33728;  load averages:  8.18,  5.96,  3.02                                        up 0+00:16:57  16:25:40
86 processes:  9 running, 77 sleeping
CPU:  0.4% user,  0.0% nice, 50.2% system,  0.0% interrupt, 49.4% idle
Mem: 438M Active, 73M Inact, 1086M Wired, 40K Buf, 29G Free
ARC: 109M Total, 25M MFU, 77M MRU, 244K Anon, 811K Header, 5895K Other
     56M Compressed, 139M Uncompressed, 2.49:1 Ratio
Swap: 8192M Total, 8192M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
65963 root          1  86    0    13M  3188K CPU12   12   5:11  53.97% ifconfig
99294 root          1  52    0    13M  3188K ecore_   2   0:08  52.68% ifconfig
11173 root          1  52    0    13M  3188K ecore_  11   4:09  49.72% ifconfig
5551 root          1  52    0    13M  3188K ecore_   5   6:37  48.16% ifconfig
83767 root          1  52    0    13M  3188K ecore_  11   2:16  47.35% ifconfig
86029 root          1  52    0    13M  3188K ecore_  13   2:11  45.51% ifconfig
46748 root          1  52    0    13M  3188K ecore_   8   3:09  44.06% ifconfig
55316 root          1  86    0    13M  3188K CPU14   14   0:56  43.60% ifconfig
6093 root          1  52    0    13M  3188K ecore_   5   6:36  42.95% ifconfig
54580 root          1  52    0    13M  3188K ecore_  15   0:56  41.80% ifconfig
84789 root          1  52    0    13M  3188K ecore_  11   2:11  37.66% ifconfig
92072 root          1  52    0    13M  3188K ecore_   5   0:23  37.36% ifconfig
80609 root          1  52    0    13M  3188K ecore_   1   4:33  36.12% ifconfig
29653 root          1  50    0    13M  3188K CPU9     9   1:08  35.92% ifconfig
10468 root          1  85    0    13M  3188K CPU10   10   4:11  35.58% ifconfig


Any tips on how to debug this further?
Are there any settings in tunables that I could use?