I am currently running OPNSense configured like multiple of the walkthroughs show online. After configuration, my speeds on my WAN interface (VirtIO) is almost identical to what I had with my bare metal install. The speeds on the LAN (VirtIO) interface are horrible. I have tried everything that I could think of to fix the issue. The one strange thing that I saw was:
ethtool --offload ens4f1 (WAN interface) generic-receive-offload off - fixing my LAN speeds, but cuts my WAN speeds in half. If I go and turn it back on, the WAN speed goes back to normal, but the download on the LAN download goes to almost nothing, but the upload is almost spot on. I would rather not pass my interfaces through to the VM if I don't have to.
Does anyone have a VM running on an internet connection that is 8/8 or higher? I think there is just a limitation with the virtualized interfaces not being able to handle the traffic.
I just installed ESXI to see if it had the same issues. Here is something VERY strange:
Windows VM - full speed 8/8
FreeBSD 14.2 - full speed 8/8
OPNSense - reduced speed to around half
Did you follow the tips here (https://forum.opnsense.org/index.php?topic=44159.0)?
You can also try using multiqueue in the NIC settings in Proxmox and hopefully, you have enabled RSS (https://docs.opnsense.org/troubleshooting/performance.html) (IDK if your walkthroughs cover that).
I have OPN virtualised on Proxmox. What is 8/8 ?
Quote from: cookiemonster on January 06, 2025, 11:14:56 AMI have OPN virtualised on Proxmox. What is 8/8 ?
8gb/8gb Fiber Connection
Quote from: meyergru on January 06, 2025, 10:20:38 AMDid you follow the tips here (https://forum.opnsense.org/index.php?topic=44159.0)?
You can also try using multiqueue in the NIC settings in Proxmox and hopefully, you have enabled RSS (https://docs.opnsense.org/troubleshooting/performance.html) (IDK if your walkthroughs cover that).
I switched to ESXi to see if that would help resolve some of the issues. I was able to get the WAN side of the network to show full bandwidth (8/8). I Did find something strange in the settings. When I went in and checked the boxes for hardware offloading, it seemed to remove it. When I had them unchecked, they seemed to work correctly. So that is the good part of the testing so far. When I check on a Windows VM on the LAN side, I get very slow download speeds, but my upload speeds seem on par to what I would think to see.
I was able to get the VM to now do a 5Gb up and down on the LAN side. The WAN side is similar. This is after I got my configuration moved over from my Baremetal machine. The Server is running Xeon Gold 5115. I wonder if they are just to slow to handle much more than that.
I have narrowed down a few things on the speed issue. I tried doing a full reset on the OPNSense configuration and everything worked perfect in terms of speed. I then started trying to disable some of the running services. I narrowed the issue down to Netflow Distributor. If I disable that service, my speeds go all the way up. I am wondering what is up with that service that could be causing the problem.
Possibly just a heck of a lot of load. Netflow is an incredibly expensive operation and might even be made even worse by virtual hardware. Just a guess, not verified in any setup, yet.
Quote from: Patrick M. Hausen on January 13, 2025, 12:19:33 AMPossibly just a heck of a lot of load. Netflow is an incredibly expensive operation and might even be made even worse by virtual hardware. Just a guess, not verified in any setup, yet.
I believe that you are 100% correct in that. I am debating if I should take the interfaces and move them from pass through to virtualized to see if it still works.