Q20331G9 high cpu usage at 10Gb/s

[alexandre.dulche]

Hello,

I decided to go back physical from VMware VM to a Qotom Q20331G9.
I reloaded/validated my configuration bit by bit on the new box and I'm now in the process of benchmarking performance before putting it into production.

My concern is about CPU usage when trying to push to 10 Gb/s using iperf3.

Source VM / Vlan A : iperf3 -c xxx.xxx.xxx.xxx -P 40 -t 600
Destination VM  / Vlan B : iperf3 -s
Opnsense 24.7.9_1

Through my Opnsense VM I can reach 8-9 Gbit/s and the CPU won't move up (load average stays <= 1.0).
It's running 4 vCPU out of a Ryzen 5 3600, 4 GB of RAM, 2 VMXNET3 vNIC . ESXi 8 host iwth intel x520 10 Gb/s uplinks.

Same test but through the Qotom box, I reach the same speed (which is good news) however the CPU easily reaches a load average over 30.0  making webUI and SSH hardly usable 

Test 1 :
Checksum offloading : disabled (=default)
LRO/TSO : disabled  (=default)
VLAN offloading : disabled (=default)
--> Load >= 30.0 (~95% system) & ~9 Gb/s

Test 2 :
Checksum offloading : enabled
LRO/TSO : disabled  (=default)
VLAN offloading : enabled
--> Load ~ 13.0 (~95% system) & ~9 Gb/s

I'm not running any IDS/IPS (yet), and just about a dozen FW rules on the test interfaces.
For the test the flow is going in and out of the same NIC (trunk port with both tagged VLANs)
I tried to disable stuff like netflow local collector but no improvement.

Any game changing tunables i should look into?
Or is the C3758R just too slow to sustain 10Gb/s without choking?

Any Qotom Q20331G9 owners who may share their experience with this unit ?

Thank you.

[alexandre.dulche]

Test 3 :
Checksum offloading : enabled
LRO/TSO : disabled  (=default)
VLAN offloading : enabled
vm.pmap.pti= 1 -> 0
hw.ibrs_disable= 0 -> 1
--> Load ~ 13.0 (~95% system) & ~9 Gb/s

No change compared to test #2 where hardware offloading seems to have helped quite a bit.