Bad (?) performance on KVM with OPNsense

[bananowy_bartosz]

Hi!

On my Host, os Clear Linux 40440, 32G RAM intel i3-9300T on E3C246D4U MOBO I have a VM with OPNsense 23.7.9-amd64 instance, with 2cores, 2G of ram, Intel Corporation I210 Gigabit Network Connection (rev 03) through  PCIe passthrough for WAN and single virtio NIC connected to linux bridge for LAN
routes are configured in such a way that from host they go through bridge to vm and to WAN.

In my tests I'm downloading 10 files, 10G each from some hosting, maxing out my WAN connection (600mbps down)

The thing that I noticed is that CPU usage reporting on my host is about 160% (for VM process, so almost maxed out the capacity of VM) in the same time, OPNsense dashboard is showing CPU usage at around 20% (from 7 to 27%)

I think that those results are odd, is it expected to see such a difference in CPU usage between dashboard and host? And more importantly, 160% for 600mb is in my opinion quite huge CPU usage, is it normal or should CPU be used less and I have a problem with my config and need to investigate more

Also iperf between OPNsense and host maxes out at around 20gbps so the issue can be seen only when accessing WAN

Br, BB

[bananowy_bartosz]

I've checked, this is not an OPNsense problem but a virtualization problem, because of some bizzare reason, NIC passthrough sometimes works like $%^# and the performance is much worst then with virtio driver... Switching to full virtualization and maybe in the future I'll find the answer