Sanity check for N100 / i226 on a 2000 / 2000 line

[Ozymandias]

I've just upgraded from a PPPoE based 900 / 110 full fibre contract to a DHCP based 2000 / 2000 with YouFibre.

I get 2000 / 2000 with the supplied router using Windows 11 and Ookla speedtest.

I get 1400 / 1400 with my N100 i226 based router both from bare metal OPNsense cli and Windows 11 (direct connection to the router).

I get similar 1400 / 1400 iperf3 numbers to and from Win11 to OPNsense (and over 2000 both ways to Unraid on the network)

I was hoping that my router would run at full 2000 / 2000 speed. Am I being unrealistic?


System:

CWWK / Topton CW-ADLN-5L Ver: 1.0 Fengshang Edition

Bios Build Date and Time - 04/26/2023 14:02:13
Bios settings:
• ASPM disabled on all ports
• PCIe speed set to Auto on all ports

N100

16GB DDR5 Ram
1TB Samsung 980 NVMe

5 port Intel I226-V
  EEPROM V2.17-0 eTrack 0x80000303

####
No line shaping or IDS/IPS.

Interfaces:
• Disable hardware checksum offload - checked
• Disable hardware TCP segmentation offload - checked
• Disable hardware large receive offload - checked
• VLAN Hardware Filtering - Disable VLAN Hardware Filtering

Tunables:
dev.igc.*.fc = 0
net.inet.rss.enabled = 1
net.inet.rss.bits = 2

net.isr.dispatch = deferred
net.isr.bindthreads = 1
net.isr.maxthreads = -1

/boot/loader.conf.local
hw.pci.enable_aspm = 0

[nero355]

I was hoping that my router would run at full 2000 / 2000 speed.

Am I being unrealistic?

What is the CPU LOAD like during those benchmarks ??


Have you tried simply downloading something instead of Ookla Speedtest.net benchmarks ?
Or any Public iPerf3 Servers ?

[Ozymandias]

Peaks at about 40%.

iperf3:
1.48 Gbits/sec from the router to a public server.
1.28 Gbits/sec from Win11 to the same public server.
1.60 Gbits/sec from Win11 to OPNsense iperf3 server.
2.37 Gbits/sec from Win11 to Unraid (N95) iperf3 server.

[dirtyfreebooter]

when RSS is enabled,

Code Select Expand

net.inet.rss.enabled = 1
the kernel will override dispatch and always make it hybrid (so you can leave that tunable out if you are enabling RSS), you can verify my claim with netstat -Q

Code Select Expand

net.isr.dispatch = hybrid
---

i tested a protectli VP2440 with N150 with i226 with Zenarmor and i was able to get 2.5 gbps

using the x710 10g interface without Zenarmor, i was able to get 9.46 gbps

so i would think that N100 could easily do 2 Gbps without IDS/IPS.

the tunables i used were (including installing the Intel Microcode Plugin):

Code Select Expand

dev.hwpstate_intel.0.epp = 10
dev.hwpstate_intel.1.epp = 10
dev.hwpstate_intel.2.epp = 10
dev.hwpstate_intel.3.epp = 10
dev.igc.0.fc = 0
dev.igc.1.fc = 0
hw.acpi.cpu.cx_lowest = C2
hw.ibrs_disable = 1
net.inet.tcp.soreceive_stream = 1
net.isr.bindthreads = 1
net.isr.maxthreads = -1
vm.pmap.pcid_enabled = 0
vm.pmap.pti = 0

note that i think the intel microcode plugin + vm.pmap.pcid_enabled = 0 is pretty much the gold standard with the N100/N150/N305 cpus, as they have a bug in the freebsd kernel otherwise...

i did update my i226 firmwares to v2.32 tho i dont think that mattered.

[meyergru]

If your connection was still on PPPoE, it would blame that, but it is not.

I have tested the pure routing speed of N100-based OpnSense systems between VLANs to ~6 Gbps, so that should work.

W/R to the Windows 11 system, I have seen network drivers that needed tuning to exceed 1.5 Gbps on some network adapters, like disabling interrupt moderation, enlarge buffer sizes or changing flow of control settings.