DEC4280 - low speed on multi-gig interfaces

Started by l1lz, July 17, 2025, 05:30:31 PM

Previous topic - Next topic
August 08, 2025, 09:05:08 PM #15 Last Edit: August 08, 2025, 10:36:44 PM by pizza_shrapnel
I've been struggling with this too.  Not sure if its the same issue, but I feel its helpful to share. My hardware is a Qotom with the C3808 Atom and 4 10G SFP ports, so maybe this isn't related, but it sure sounds like it might be. 

Where I see slowdown is in receiving on the interface. Just a simple iperf test, I can get 9.4Gb sending firewall->server.  But all sending from server->firewall are 1G-1.5G. Between servers i get 9+Gb both direction, on same switch.  Like others, testing with direct connection, connect to switch, using fiber , DAC, or CAT6a, different SFP modules, different ports, different server, different NIC's, same results.  And tried all kinds of tunnables.  But I'm seeing some more in other posts I'd like to test, and I'll double check my bios. 

@AdSchellevis, Sorry, I understand me creating a new account just to chime in isn't too much help, but I find it helpful. ;)  If there's anything I can test I'd be happy to report back.

Edit:
I wonder if this has something to do with this? Link PCIe speed is 1x?

pciconf -lbcevV pci0:11:0:0
cap 10[a0] = PCI-Express 2 endpoint max data 128(128) FLR RO
                 max read 512
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)

Quote from: pizza_shrapnel on August 08, 2025, 09:05:08 PM[...]
I wonder if this has something to do with this? Link PCIe speed is 1x? [...]

Could be. The SOC could have some oddities, but x1 v1 for a 10Gb interface seems unlikely. I'd run the query against the device mnemonic rather than the PCI address.

If that link speed is correct (and pciconf usually does report it correctly), then the speed would be limited at 500 MBit/s with 1 * PCIe 2.x.

My OpnSense show this for an Intel 82599ES:

    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
                 max read 512
                 link x4(x8) speed 5.0(5.0) ASPM disabled(L0s)

Yours seems to even be a toned-down PCIe 2.x version (because the base speed of PCIe 2 should be 5 GT/s, not 2.5 GT/s, which would be PCIe 1.0).

Was that for the correct device? You can list all by leaving out the pci bus id.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Can you dump sysctl -a into a post. Feel free to mask out any settings you don't want seen publicly.
Does that dec have HT enabled?

Hello @BrandyWine,
Here is our sysctl -a output:
privatebin.net
Hyperthreading is enabled.
Kind regards

Hi,

We just uploaded a new bios version, which can be downloaded from https://docs.opnsense.org/hardware/bios.html#dec4200-series and should resolve the throughput issues we have seen.


Best regards,

Ad

I confirm that the new BIOS version solved the issue for me. Many thanks!