Messages

1

24.7 Production Series / bnxt driver for Broadcom NIC cards now automatically loaded

« on: July 23, 2024, 03:21:56 pm »

Hello,

we recently tested upgrades and also fresh installs of OPNsense 24.7 RC2 with different systems.
It runs fine without any issues hardware wise with our test systems.

We discovered that the bnxt driver for Broadcom 10G/25G NIC cards will be automatically loaded when such a card ist installed. Thats very nice and saves work during install!
It's already useable if you fire up a OPNsense 24.7 RC2 boot drive and run it in live mode.

But unfortunately I didn't find any hint proving this outcome. Do you know if the bnxt driver is now directly compiled into the kernel?
I did some research on a system with 24.7 RC2 and if you call kldstat it prompts that the if_bnxt.ko is loaded.

Best regards and thanks again for your great work!
Thomas

2

24.1 Legacy Series / Re: Multi WAN setup

« on: February 07, 2024, 03:52:06 pm »

Oh okay sorry for that, I maybe got it wrong while reading your text initially.
Hmm yes mobile connections are always a bit buggy, because a LTE Modem (for example Quectel EM12-G) is kind of a computer on its own.

I recently tested it again, if you dare to unplug the SIM and insert it afterwards you might guess it would come up again and reconnect... no way you have to unplug and de-energize the whole machine and then boot it up again to get it working again.

The fact that you have to delete it completely... I have no clue...

3

24.1 Legacy Series / Re: Multi WAN setup

« on: February 07, 2024, 03:29:38 pm »

What I can say is that if you stick to this article it works fine: https://docs.opnsense.org/manual/how-tos/multiwan.html
We also did our own tests and created a wiki article: https://www.thomas-krenn.com/de/wiki/OPNsense_Multi_WAN
I hope that helps.

Best regards,
Thomas

4

Hardware and Performance / Re: Intel Atom P5000 / P5322 Support

« on: November 17, 2023, 01:34:55 pm »

A move to FreeBSD 14 would be really great, because we discovered that now with the FreeBSD 14-Release (released a couple of days ago) the Atom Board with C5315 from Supermicro works just fine (A3SPI-4C-LN6PF). The same board we tested with a earlier version of FreeBSD 14 didn't boot at all (as I already replied in this thread).
So we thought let's give it a try and rerun the test with the current FreeBSD 14 version, as the test was nearly half a year ago. And as mentioned above, absolutely fine and no issues until now.

So it would be amazing if OPNsense would upgrade to a FreeBSD 14 Kernel next year!

Cheers,
Thomas

Here for confirmation a excerpt of "dmesg -a":

Code: [Select]

---<<BOOT>>---
Copyright (c) 1992-2023 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 05:57:23 UTC 2023
    root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152)
VT(efifb): resolution 1024x768
CPU: Intel Atom(R) C5315 processor (2400.00-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x80667  Family=0x6  Model=0x86  Stepping=7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x4ff8ebff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x101<LAHF,Prefetch>
  Structured Extended Features=0x2394b2c3<FSGSBASE,TSCADJ,FDPEXC,SMEP,ERMS,PQM,NFPUSG,PQE,RDSEED,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
  Structured Extended Features2=0x1a400124<UMIP,WAITPKG,GFNI,RDPID,CLDEMOTE,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0xfc000400<MD_CLEAR,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x20c6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16228294656 (15476 MB)
Event timer "LAPIC" quality 600


5

23.7 Legacy Series / Re: Broadcom P225P driver - vlans no longer work

« on: August 25, 2023, 12:11:49 pm »

We did some further tests and the tests so far went well.
We already did some tests with ping and currently we are currently running iperf3 tests.
No issues so far. We are not done yet, but I wanted to keep you updated.

I also don't have much doubt about shipping this patch with a future kernel, would be great.

Cheers,
Thomas

6

23.7 Legacy Series / Re: Broadcom P225P driver - vlans no longer work

« on: August 23, 2023, 04:13:00 pm »

First update: We installed the Kernel and the problem seems fixed, VLANs work!
The interface was accessible via ping. But we will go on and test the new Kernel further in the next days.
Would be fantastic if we already solved the problem with this Kernel.

7

23.7 Legacy Series / Re: Broadcom P225P driver - vlans no longer work

« on: August 23, 2023, 03:41:26 pm »

Hi Franco,

wow very nice, we'll test the Kernel with the cards and give you feedback.
Hopefully it solves the problem, would be fantastic.

Cheers,
Thomas

8

23.7 Legacy Series / Re: Broadcom P225P driver - vlans no longer work

« on: August 23, 2023, 11:07:16 am »

Hello everyone,

we internally tested also all of our Broadcom cards and the issue affects not all NICs.
Only the cards with the BCM574xx chip are affected from this, we also wrote a Wiki article about that issue: https://www.thomas-krenn.com/en/wiki/Broadcom_BCM574xx_VLAN_problem_under_FreeBSD_13.2_with_bnxt_driver
We are also in contact with Broadcom and Kevin Bowling already made a patch: https://reviews.freebsd.org/D41558

Best regards,
Thomas

9

Hardware and Performance / Re: Broadcom P425G not working (driver) on OPNsense 23.1

« on: June 28, 2023, 02:40:53 pm »

We tested the new kernel, works fine!
Driver is updated to 2.20.0.1. We did some quick tests, connected bnxt2 (first NIC on the P425G) to the network and it has received an address via dhcp.

root@OPNsense:~ # sysctl -a | grep -E 'dev.*.iflib.driver_version' -A 5
dev.bnxt.2.iflib.driver_version: 2.20.0.1
dev.bnxt.2.%domain: 0
dev.bnxt.2.%parent: pci10
dev.bnxt.2.%pnpinfo: vendor=0x14e4 device=0x1751 subvendor=0x14e4 subdevice=0x4250 class=0x020000
dev.bnxt.2.%location: slot=0 function=0 dbsf=pci0:1:0:0
dev.bnxt.2.%driver: bnxt

Quad Port P425G 25G Card (# sysctl -a |grep -i broadcom):

bnxt2: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc030000-0x303dbc03ffff,0x303dbb000000-0x303dbbffffff,0x303dbc058000-0x303dbc05ffff irq 88 at device 0.0 numa-domain 0 on pci10
bnxt3: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc020000-0x303dbc02ffff,0x303dba000000-0x303dbaffffff,0x303dbc050000-0x303dbc057fff irq 89 at device 0.1 numa-domain 0 on pci10
bnxt4: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc010000-0x303dbc01ffff,0x303db9000000-0x303db9ffffff,0x303dbc048000-0x303dbc04ffff irq 88 at device 0.2 numa-domain 0 on pci10
bnxt5: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc000000-0x303dbc00ffff,0x303db8000000-0x303db8ffffff,0x303dbc040000-0x303dbc047fff irq 89 at device 0.3 numa-domain 0 on pci10

Dual Port 100G P2100G Card:

bnxt6: <Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbf010000-0x303dbf01ffff,0x303dbe000000-0x303dbeffffff,0x303dbf028000-0x303dbf02ffff irq 90 at device 0.0 numa-domain 0 on pci11
bnxt7: <Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbf000000-0x303dbf00ffff,0x303dbd000000-0x303dbdffffff,0x303dbf020000-0x303dbf027fff irq 91 at device 0.1 numa-domain 0 on pci11

10

Hardware and Performance / Re: Broadcom P425G not working (driver) on OPNsense 23.1

« on: June 27, 2023, 04:10:13 pm »

Hi Franco,

thanks for the new sets!
I will test it tomorrow with the Broadcom cards and give feedback.

Best regards,
Thomas

11

Hardware and Performance / Re: Broadcom P425G not working (driver) on OPNsense 23.1

« on: June 27, 2023, 03:03:37 pm »

Hello,

yes these Broadcom Cards don't work with OPNsense 23.1, we discovered this issue internally some time ago.
The FreeBSD driver for these quite new cards is too old to support them, but Broadcom updated it and with FreeBSD 13.2-Release these cards are recognised and work quite good.
I only tested it with plain FreeBSD 13.2-Release, because OPNsense is based on 13.1-Release.
But Franco tweeted in May that he successfully booted OPNsense with 13.2, so I hope future OPNsense 23.7 is based on 13.2, that would fix the issue with these cards.

Best regards,
Thomas

Output from FreeBSD 13.2-Release:

root@:~ # sysctl -a |grep -i broadcom
 
bnxt2: <Broadcom BCM57504 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet> mem 0x303dbc030000-0x303dbc03ffff,0x303dbb000000-0x303dbbffffff,0x303dbc058000-0x303dbc05ffff irq 88 at device 0.0 numa-domain 0 on pci11

--> BCM57504 is the IC of the P425G

New driver version in FreeBSD 13.2-Release:
root@:~ # sysctl -a | grep -E 'dev.*.iflib.driver_version' -A 5
dev.bnxt.7.iflib.driver_version: 2.20.0.1
dev.bnxt.7.%domain: 0
dev.bnxt.7.%parent: pci12
dev.bnxt.7.%pnpinfo: vendor=0x14e4 device=0x1750 subvendor=0x14e4 subdevice=0x2100 class=0x020000
dev.bnxt.7.%location: slot=0 function=1 dbsf=pci0:2:0:1
dev.bnxt.7.%driver: bnxt

12

Hardware and Performance / Re: Intel Atom P5000 / P5322 Support

« on: June 13, 2023, 10:41:52 am »

Hi,

we are currently evaluating a Supermicro A3SPI-4C-LN6PF motherboard with OPNsense. Fails also with error 19, so I came across this message. Tried to install FreeBSD, similar issue "Mounting from ufs:/dev/ufs/FreeBSD_Install failed with errror 19."Also tried the FreeBSD 14.0 Current code. Tried USB 2.0 flash drives, IPMI ISO mount, DVD drives. Always the same error. Seems that the architecture is not supported yet. We already reported the problem to Supermicro.

Best regards,
Thomas

13

18.1 Legacy Series / Re: [Workaround] Segmentation fault while installing 18.1 on a new hardware

« on: June 28, 2018, 02:55:40 pm »

Hi Franco,

I did the tests for Werner and you'll find the output of dmesg and installer.log attached to my post.
I hope these two files help to fix the problem.

Cheers,
Thomas