Slow throughput

[Liboicl]

So, I recently upgraded to Gigabit internet and my old router with a built-in firewall wasn't cutting it. The CPU maxed out and would only give around 400Mbps max. I decided I needed to just put an actual firewall in front of a switch. I started fiddling with OPNsense and have it working now, but it's even slower.

I searched around and it definitely seems like this shouldn't be an issue. I am getting ~150Mbps now when using OPNsense. I am running OPNsense 21.1.3_3-amd64 through qemu with KVM on a home server. The CPU never gets above 30% under heavy load. I am using the default settings on everything, except a couple of static DHCP leases.

By the way, if I connect my PC straight to my modem I am getting ~980Mbps.

Any hints and or points in the right direction will be greatly appreciated.

EDIT: Switching to virtio from e1000 has increased the performance quite a bit. I'm now getting ~700Mbps that's even directly to iperf3 on the opnsense machine. If I run iperf3 to a Linux virtual machine I get a throughput of ~1.7Gbps, so it's still not where it needs to be. Any suggestions on an optimal configuration?

[lfirewall1243]

What hardware ?

[Liboicl]

I've got 2 NC382i integrated chipsets for a total of 4 ports. I have 2 ports bridged over virtio now. I've got dual Intel X5650s with all 24 cores allocated to the VM. I'm running it with 8GB out of my 64GB total RAM.

I've done a bit more troubleshooting. Someone on Reddit suggested a single core may have been getting maxed out, but using top shows that a single core never gets above 25%. I may end up just switching to IPFire, as Linux-based VMs don't seem to have this issue. I would rather figure this out, though.

I tested with IPFire and I am getting 2Gbps throughput. So obviously it is some issue with configurations. I am going to boot HardenedBSD directly and test it.

EDIT: Also of note, IPFire even under load running 2Gbps iperf3 transfer or 1Gbps speed tests barely gets over 2% CPU usage. I'm moving to it for now, but I'm still interested in trying to get opnsense to work.