Slow download speed compared to directly connected to modem

[andrema2]

Hi,

I used to have a 350 mbits Download /150 mbits upload bandwidth with my local provider. I just changed it to 700/350. Behind the firewall I cannot go over 350 as I did before. If I connect directly on the ISP modem I can reach over 700 download and 350 upload using the same OOKLA test.

All my interfaces are 1gbits, I'm running it on a proxmox server. I don't know what can I do to be able to achieve the same speed behind the fw as I do directly connected.

IPS is disabled.
Any ideas ?

[JasonJoel]

Ideas....


1. Running your perimeter security device in a VM is a very poor security practice.
2. If you must run in a VM, review what virtual hardware devices you are using in the VM for the NICs. You may have chosen a high compatibility, but lower performance, option. Paravirtualized is typically higher performance than Intel or VMware options in Proxmox.
3. Review CPU and memory configuration of the VM. Increased throughput usually required some amount of increased CPU.
4. Review what the hardware NICs you have in the host. Maybe that is all the throughput they can handle when used in a virtualized environment, not that uncommon with Realtek NICs (although that is a lot less true today than it used to be in the past).

[andrema2]

Yes, I need to run it as a VM.

I have it set on a host with 8 x 12th Gen Intel(R) Core(TM) i3-12100T and 20gb of RAM, all interfaces are VirtIO. The processor and memory usage seems very low. Any other suggestion ?

[JasonJoel]

Not from me, no. Sorry. I would get physical hardware.

[patient0]

- How to you connect to the ISP? PPPoE? DHCP?
- What virtualisation software do you use? (I sometimes can't read)
- How many vCPUs and RAM did you assign to the VM
- Any traffic shaping involved where you set a upload/download limit?
- Hardware Offloading disabled?

[andrema2]

I'm connected to the ISP using DHCP.
There is 8 vCPUs and 20GB of RAM for this VM.
All hardware offloading is disabled.
There is no traffic shapping set

[patient0]

That should more than enough (I'd say 4 CPUs and 8 GB would do it).

But:
- How do you connect the VM to your ISP? A bridge in Proxmox with the physical NIC added to it and the VM connected to this bridge?
- How do you test it, from OPNsense itself (you shouldn't do that) or from a client (of course connected to OPNsense)?
- Is the upload speed also stuck at 150Mbit?

[andrema2]

Yes, I'm connected to a bridge in Proxmox and it is connected to the physical NIC. The WAN Bridge and NIC is only used by the Opnsense VM.

I tried to test if from a client and also from the Opnsense with the same results. Both upload and download is topping at 350mbits. The upload seems correct. Only the download that is lower than the actual speed.

[cookiemonster]

Are you not able to pass through the NIC to the VM? That' how I've been running my installation recently and get most of my bandwidth of 510 mbps down. I get it all actually, but IPS takes its share.

[patient0]

...
I tried to test if from a client and also from the Opnsense with the same results. Both upload and download is topping at 350mbits. The upload seems correct. Only the download that is lower than the actual speed.

Mmmhh, I run OPNsense on XCP-NG on a fitlet2 (Intel J3455) with 4 vCPU and 4GB RAM with PPPoE (which is singel-threaded) on 400/90Mbit. You're config must be plenty enough. Maybe try with less vCPUs? In my tests sometimes too many vCPUs where not helpful. If you have the patients start with 2 vCPUs.

I assume you're modem is in bridge mode? And you did reboot the modem and OPNsense. That's of course not a very technical approach but it doesn't hurt .