opnsense with Intel X710-DA2 Bandwidth low

[wondaah]

Hi everyone

I have a Proxmox Server with a 6 Core 12 Thread 4.7GhZ AMD Ryzen 5 7600X and 32GB DDR5 RAM. In that Proxmox Server I have virtualised an opnsense that does NAT, DHCP and inter-vlan Routing.

My Problem is that I only get around 2.5 - 3 Gbit/s on my 10Gbit/s interface. It is connected directly to my ISP where I work at as a Network Engineer, however as I work at an ISP I do not work with Firewalls that much because they live at our customers.

PCI-E is fast enough and has enough lanes. CPU should be fast enough as well.

I don't know if I'm missing any tunables or options or is there something else I can check?

Edit: Sorry I'm using the Intel X710-DA2  not the Mellanox.

Edit 2: Maybe something I could try would be PCI Passthrough into the VM that is running opnsense. But I would rather not do that because it involves a lot of work again.

[meyergru]

Running a virtualized NIC is certainly not the best option performance-wise. Modern NICs have special hardware involving buffers, DMA and things like that to make them perform better.

Once you virtualize that NIC, many optimizations w/r to buffering are essentially dropped. The best way to go forward would be to pass those NICs through, which should be fairly easy as OpnSense allows you to reassign interfaces on the CLI, which is easily possible via Proxmox console access to the VM.

Otherwise, you could try if another emulation would do better (there are several types available), however, those could also result in different interface names on the VM.

[johndchch]

first thing to try would be running opnsense bare-metal rather than virtualised - it's the only way to tell if the issue is the hypervisor and your use of vmnics, or you're actually running into hardware limits ( I presume you've already watched cpu load whilst running tests to check you're not bottlenecking on a single core)