Messages

On the Proxmox host, stop the VM and add a virtual serial port:
bash

qm stop <VMID>
qm set <VMID> -serial0 socket
qm start <VMID>

In the FreeBSD VM, configure the serial console:
bash

sudo sysrc console="comconsole,vidconsole"
sudo sysrc boot_multicons="YES"
sudo sysrc boot_serial="YES"
sudo sysrc comconsole_speed="115200"

Need to look deeper.

0x1458 is indeed the AMD EPYC integrated 10-GbE controller used by FreeBSD's axp driver. The important detail is that FreeBSD's driver was added as an iflib-based driver, and the hardware exposes separate RX DMA queues and management counters. If traffic works but RX counters remain at zero, this is likely a driver/statistics-path problem rather than PCI identification.

EPYC's do have various implementations of the PHY, but I see most (the newer stuff) use the axp driver. The pciconf shows a device ID that supposedly uses the newer AXP code, but technically still from the amd-xgbe module.

I suspect an iflib issue.

What AMD EPYC is it exactly?


https://cgit.freebsd.org/src/commit/?id=7113afc84c0

maybe
ifconfig <interface> media 1000baseSX

Is the cpu being fully offloaded to the NIC now?

You can try disabling this feature for individual NIC functions using sysctl.
Or use ifconfig ax1 -tso4 -tso6

So 1st try that.

To me it just sounds like a driver issue.

But if I connect the USB.....

??
Why exactly are you plugging in a mifi pro m4 into the fw with USB ? For batt charging ? Why ?

Use a StarTech USBSCHAAMF data blocker.

Why it's messing with the fw, look at logs. Perhaps the device is loaded with malware?

Several things come to mind.
1) iperf3 (on the router) is not a tool for testing bandwidth of a router. We dont run apps on the fw, we use the device as a fw/router.
2) run two instances of iperf3 using --affinity switch
3) scheduling of threads can be an issue, the kernel wants to move threads around cpu's (scheduler attempt to balance). As-is, manipulating the OS so that the cpu's wanted for just iperf3 have only iperf3 threads on those cpu's is not easy doing. Threads that move take time because the cache on new cpu needs to fetch from memory. One trick here is, if you want 8 threads and eliminate the cache timing issue, run 8 instances of iperf3 where each is bound to a unique cpu. You can also tune the Time Quantum settings for the default ULE scheduler.

I bet you can get above 8G in testing, but this will not be indicative of actual fw/router bandwidth. The next step I would do, test the device as just a  router (tuned iperf3 server and client on each side of the router, dec in middle).

Run the same test as you did, note the results, then just run "sysctl net.inet.tcp.sendbuf_max=16777216" then re-run the testing again, any difference?

All that said, there's tcp RTT issues not accounted for, which impacts measured bandwidth.
https://www.pinglabz.com/how-to-use-iperf3/
https://www.pinglabz.com/iperf3-parallel-streams-tcp-window/

[the actual update fails]

One thing that may be relevant: NVMUpdate does recognize the controller as 8086:125F ("Client Foxville with Empty Flash Image") and accepts the 125B I226-LM image/config, but the actual update fails at the flash access stage. The log reports:

run your nvmupdate command again, but add ;echo $?
add semicolon with echo $?

Lets see what that looks like.

What Intel docs describe the EFI Lan driver being in the NVM bin file?

I would put the Linux tools (and the bin files) onto a bootable liunx USB, work it from that angle.

Make sure there's no BIOS settings that prevent writing to PCIe devices.

Legacy bios from mobo chip?
UEFI is protected by TPM?

Take the EEPID and REPLACES out of the cfg file, these are optional cfg parameters.

When it fails, what is the exit code?
https://edc.intel.com/content/www/us/en/design/products/ethernet/adapters-and-devices-user-guide/intel-ethernet-nvm-update-tool/#command-line-parameters

What does the inventory output say? Why is the device tagged with a Micro Star subvendor?

@and,

Please list your cfg file.

Use some PCB 2.4GHz yagi antennas, just make sure they stay pointed in proper direction. This will significantly increase range.

This one is interesting, no VSWR spec, so I ordered one just to see what it looks like on my VNA.

https://www.amazon.com/dp/B0H3ZYM661

make the OPNsense 26.7.2 version with 15.1-rc3

Dude... you know 15.1 is newer than the RC3 release for 15.1 right ?!

And again : The problem are FreeBSD 14.x to 15.x upgrades and not fresh installs of FreeBSD 15.x a.k.a. OPNsense 26.7.x :)

15.1 Release is June 16th (a lie on their site, https://www.freebsd.org/releases/)
15.1 is actually June 12th (https://download.freebsd.org/releases/amd64/15.1-RELEASE/)
RC3 is June 6th.

What's the diff?

My point is, make the OPNsense 26.7.2 version with 15.1-rc3. So when I upgrade into 26.7.2 from 26.x less than .7, I don't need to do any manual bootloader copying.

But as I noted, that URL I gave, is talking about rc3, but then goes onto say "copy in the fixed bootloader". If rc3 already has the fixed bootloader then why would I be copying it in?