(SOLVED) Proxmox and Gigabit WAN>LAN

[bunchofreeds]

Hi all,

I'm after some help with achieving the following goals:

-Near Gigabit throughput WAN>LAN
-OPNsense as a Proxmox Guest
-Running using virtio adapters for live migration
-Ideally running IPS and NTOPNG

This is for a home setup to sit in front of a home LAN with loads of IoT and some general IT lab work.
It's a real budget setup and I don't wan't to go to crazy on upgrades.

Hardware I have is:

2x
HP desktop 8300 SFF
Intel i7-3770
32GB RAM
Onboard Intel Gig Nic
Intel 82576EB dual port Gig Nic

48 port managed Gigabit switch

900/400 WAN via PPPoE on VLAN 10

I want to run OPNsense virtual on Proxmox as I'd like to be able to live migrate this when doing any work on the host. I have had this running in the past successfully but only had 100/20 WAN at that time.
The most I could get through this setup then was ~600Mbit. This was after trying most everything including tunables.

Can anyone help with where the bottle neck is with not being able to reach near Gigabit?
I have an old Linksys router running Tomato firmware that can reach 900+ Mbps so the possibility exists.
Would upgrading to 10Gbe adapters (e.g. Intel x540-T2) give me the head room or is the limit somewhere else, maybe the CPU?

Thanks for any helpful advice


 

[bunchofreeds]

Just an update on my progress.

It would seem I can only get ~470 Mbits when the Reporting>Traffic graph is being displayed.
I can get ~900Mbps + when this is not being used (Maximum from my ISP).

Probably a bug I'm guessing...

I'm using Proxmox > linux Bridge > Guest Virtio + Vlan > PPPoE in OPNsense.

Lol, this had me trying out a LOT of options.

[opnfwb]

I don't personally use the PPPoE function however, I have read that it is single threaded. Is one of the threads on the OPNsense VM being CPU bound during the speed tests?

Still, with that hardware and virtualization, 900mbps isn't too bad.

[bunchofreeds]

Yes its a great setup really.
It's super fast and really quite inexpensive. The hardware is easy to find as they were very popular as office desktops, so easy to replace. Also designed to run 24/7. Only uses about 50 watts when servicing basic requirements of a few VM's

I get maxed out gigabit through this setup with little impact to CPU etc. All offloads disabled.
It's also capable of SR-IOV for a dual NIC when in PCIE slot 1x, but you then can't easily migrate the VM.

Another note is to run speed tests using the speedtest.net CLI tool and not their web based.
The web based when run remotely has issues recording the correct speed I found.