OPNsense 4x slower than PFSense on same hardware

[opnfwb]

Yes, I see what you mean with pfSense being able to sustain a higher throughput under the same circumstance.

Unfortunately, I'm not sure if this is useful data because it isn't telling us how fast either solution can actually route packets. In actual use, we'd be using pfSense or OPNsense as a firewall/router setup and we would want to see how quickly they can push traffic through themselves, rather than serving traffic directly through one interface.

I suppose the last things I would check would be to verify that OPNsense and pfSense both have OpenVMTools installed/running. And run a 'top -aSCHIP' on an SSH console for both of them and see what their CPU usage is when running your transfer test. Watching them under load may reveal a bottleneck, especially on the OPNsense router since that one seems to be under performing in your tests.

Finally, if you feel like digging in a bit more, I would recommend doing a test using Proxmox client and server VMs, and have two switches inside Proxmox. One can be used for the WAN port and one switch is a private switch with no physical uplinks, we'll use this for the LAN port. Doing this method will allow you to simulate actual routing performance of both solutions and you won't need an extra physical client.

[mimugmail]

One of my clients pushes 2,6Gbit with ESX6.7 and vmxnet3. With Chelsio 40G cards I was able to route 22Gbit