OPNsense Forum
English Forums => Hardware and Performance => Topic started by: Hans Peter Graf on June 22, 2023, 09:21:05 pm
-
I recently installed a Broadcom P425G (4x 25Gbe) on a hardware running OPNsense 23.1. Unfortunately, there seems to be a problem with the drivers and no ethernet card is shown (ifconfig -a).
I should mention that on Linux the P425G Ethernet card works, and that on Linux I was able to install the latest P425G firmware [1].
OPNsense (13.1-RELEASE-p7) has a driver/module: if_bnxt.ko . With this module loaded, no network device is found. Furthermore, I tried to compile the latest version of the if_bnxt.ko module from [1]. Even after unload the bundled driver and loading the new driver, no Ethernet device was found.
Do you have any suggestion ?
Here a portion of the output of pciconf -lbcevV :
none6@pci0:1:0:0: class=0x020000 rev=0x11 hdr=0x00 vendor=0x14e4 device=0x1801 subvendor=0x14e4 subdevice=0x4250
vendor = 'Broadcom Inc. and subsidiaries'
device = 'BCM57504 NetXtreme-E Ethernet Partition'
class = network
subclass = ethernet
bar [10] = type Prefetchable Memory, range 64, base 0x6045070000, size 65536, enabled
bar [18] = type Prefetchable Memory, range 64, base 0x6044000000, size 16777216, enabled
bar [20] = type Prefetchable Memory, range 64, base 0x60450b8000, size 32768, enabled
cap 01[48] = powerspec 3 supports D0 D3 current D0
cap 03[50] = VPD
cap 11[a0] = MSI-X supports 74 messages
Table in map 0x20[0x0], PBA in map 0x20[0x4000]
cap 10[ac] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
max read 4096
link x16(x16) speed 16.0(16.0) ClockPM enabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
ecap 0003[13c] = Serial 1 84160cfffe648250
ecap 0004[150] = Power Budgeting 1
ecap 0002[160] = VC 1 max VC0
ecap 000b[180] = Vendor [1] ID 0000 Rev 0 Length 32
ecap 0018[1b0] = LTR 1
ecap 000e[1b8] = ARI 1
ecap 0010[1c0] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
0 VFs configured out of 8 supported
First VF RID Offset 0x0100, VF RID Stride 0x0001
VF Device ID 0x1806
Page Sizes: 4096 (enabled), 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 4194304
ecap 0017[230] = TPH Requester 1
ecap 0019[300] = PCIe Sec 1 lane errors 0
ecap 001f[200] = Precision Time Measurement 1
ecap 0026[358] = Physical Layer 16.0 GT/s 1
ecap 0027[388] = Lane Margining at Receiver 1
PCI-e errors = Correctable Error Detected
Unsupported Request Detected
Corrected = Advisory Non-Fatal Error
VPD ident = 'Broadcom NetXtreme E-Series Quad-port 25Gb SFP28 PCIe Ethernet Adapter'
VPD ro PN = 'BCM957504-P425G'
VPD ro MN = '14E4'
VPD ro V0 = '226.1.107.1'
VPD ro V1 = '226.0.135.0'
VPD ro V3 = '226.0.145.1'
VPD ro V6 = '226.0.135.0'
VPD ro V7 = '0.0.0'
VPD ro V9 = '0.0.0'
VPD ro VB = 'REV016DEV869DEV917'
VPD ro SN = 'P425G2215003XXXX'
[1] https://www.broadcom.com/products/ethernet-connectivity/network-adapters/p425g
-
Hello,
yes these Broadcom Cards don't work with OPNsense 23.1, we discovered this issue internally some time ago.
The FreeBSD driver for these quite new cards is too old to support them, but Broadcom updated it and with FreeBSD 13.2-Release these cards are recognised and work quite good.
I only tested it with plain FreeBSD 13.2-Release, because OPNsense is based on 13.1-Release.
But Franco tweeted in May that he successfully booted OPNsense with 13.2, so I hope future OPNsense 23.7 is based on 13.2, that would fix the issue with these cards.
Best regards,
Thomas
Output from FreeBSD 13.2-Release:
root@:~ # sysctl -a |grep -i broadcom
bnxt2: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc030000-0x303dbc03ffff,0x303dbb000000-0x303dbbffffff,0x303dbc058000-0x303dbc05ffff irq 88 at device 0.0 numa-domain 0 on pci11
--> BCM57504 is the IC of the P425G
New driver version in FreeBSD 13.2-Release:
root@:~ # sysctl -a | grep -E 'dev.*.iflib.driver_version' -A 5
dev.bnxt.7.iflib.driver_version: 2.20.0.1
dev.bnxt.7.%domain: 0
dev.bnxt.7.%parent: pci12
dev.bnxt.7.%pnpinfo: vendor=0x14e4 device=0x1750 subvendor=0x14e4 subdevice=0x2100 class=0x020000
dev.bnxt.7.%location: slot=0 function=1 dbsf=pci0:2:0:1
dev.bnxt.7.%driver: bnxt
-
The latest 13.2/23.7 OS sets are:
# opnsense-update -zbkr 23.7.d_24
# opnsense-shell reboot
It would be nice to get a confirmation if these have been fixed indeed.
Cheers,
Franco
-
Hi Franco,
thanks for the new sets!
I will test it tomorrow with the Broadcom cards and give feedback.
Best regards,
Thomas
-
We tested the new kernel, works fine!
Driver is updated to 2.20.0.1. We did some quick tests, connected bnxt2 (first NIC on the P425G) to the network and it has received an address via dhcp.
root@OPNsense:~ # sysctl -a | grep -E 'dev.*.iflib.driver_version' -A 5
dev.bnxt.2.iflib.driver_version: 2.20.0.1
dev.bnxt.2.%domain: 0
dev.bnxt.2.%parent: pci10
dev.bnxt.2.%pnpinfo: vendor=0x14e4 device=0x1751 subvendor=0x14e4 subdevice=0x4250 class=0x020000
dev.bnxt.2.%location: slot=0 function=0 dbsf=pci0:1:0:0
dev.bnxt.2.%driver: bnxt
Quad Port P425G 25G Card (# sysctl -a |grep -i broadcom):
bnxt2: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc030000-0x303dbc03ffff,0x303dbb000000-0x303dbbffffff,0x303dbc058000-0x303dbc05ffff irq 88 at device 0.0 numa-domain 0 on pci10
bnxt3: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc020000-0x303dbc02ffff,0x303dba000000-0x303dbaffffff,0x303dbc050000-0x303dbc057fff irq 89 at device 0.1 numa-domain 0 on pci10
bnxt4: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc010000-0x303dbc01ffff,0x303db9000000-0x303db9ffffff,0x303dbc048000-0x303dbc04ffff irq 88 at device 0.2 numa-domain 0 on pci10
bnxt5: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc000000-0x303dbc00ffff,0x303db8000000-0x303db8ffffff,0x303dbc040000-0x303dbc047fff irq 89 at device 0.3 numa-domain 0 on pci10
Dual Port 100G P2100G Card:
bnxt6: <Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbf010000-0x303dbf01ffff,0x303dbe000000-0x303dbeffffff,0x303dbf028000-0x303dbf02ffff irq 90 at device 0.0 numa-domain 0 on pci11
bnxt7: <Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbf000000-0x303dbf00ffff,0x303dbd000000-0x303dbdffffff,0x303dbf020000-0x303dbf027fff irq 91 at device 0.1 numa-domain 0 on pci11
-
ok great :)