Low throughput on LAN

[matokarlukas]

Hello, i use OPNsense like as a virtual server with these parameters: CPU: Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz (4 cores),  memory RAM: 4GB+Swap. All our network adapters are join with 10GB/s (f.e.    public Ethernet 10Gbase-T <full-duplex>ip). I tried to iperf to wan interface with relatively (in comparison with lan speedtest) good results (always more than 3GBs). My LAN computer is connected with 1Gb/s. But when i try to speedtest (with the closest servers), i cant measure more than 300-320Mb/s down/up. In our network, i also tried simple virtual router (with ipconfig) and speed are always near 1Gb/s and the same result when i try it with device Unifi dream machine Pro. Where can be a problem? I installed opnsense because it looks to be a great device for our network.

[bartjsmit]

Which hypervisor? Which virtual NIC? Are you testing with a bare bones OPNsense (NAT only)? Anything running hot (CPU/Mem/Disk/Network) on the hypervisor monitoring?

[matokarlukas]

hypervisor is KVM (proxmox), virtual NIC is Virtio. Yes, i am testing without any firewall shaping rules (just outbound NAT). Nothing is running hot during the network activity (monitored by HV and zabbix)

[testo_cz]

Just to double-check how your throughput numbers were measured:

Not sure , if you did run iperf3 on your OPNsense VM , but it is strongly discouraged to do so, namely because of throughput results are inconclusive.
iperf3 data should always go through the OPNSense , e.g. PCinWan <->  OPN <-> PCinLan.
With default OPN there is Outbound NAT and so it means PCinLan (iperf3 -c the-wan-IP ) opens the connection to PCinWan (iperf3 -s -B the-wan-IP ).

To get some decent measurements, use more TCP streams and longer than default test periods (e.g.: -t 60 )
iperf3 -P <num> ... to increase No. of TCP , 2 ,5 ,or 10
and
iperf3 -R ... with or without , to test also the other direction

Thanks