Topics

I'm in the process of migrating my OPNsense installation from one virtualized host to another (both Proxmox).  On the old host, I have 4 physical network ports, one of which is dedicated as an interface for some "public-facing" servers (subnet 192.168.200.0).  On my new host, I only have two network ports, which I didn't expect to be a problem, thinking I could easily just convert the "physical" Interface over to a new VLAN under the main physical LAN interface.  I already have 3 other VLANs setup this way (Main/VLAN 2, Isolated/VLAN 10, and Guest/VLAN 20) with no issues.  I decided it was best to make this switch before migrating over to my new Proxmox host to keep things as clean as possible.  Unfortunately, when I create a new VLAN (vtnet1_vlan200), and assigned it to my "Public" interface, my servers become unreachable (outside of the console within Proxmox) and have no internet connectivity.  I switch back to the physical interface (vtnet2) and it goes back to working just fine. Things I've tried/checked (multiple times):

- Setting up the new VLAN and reassigning the interface via both the web GUI and console
- Double-checking firewall rules pertaining to this interface
- Disabling/re-enabling firewall rules to see if that resyncs anything
- Checked firewall logs and don't see anything that seems to indicate connections are blocked, but I could be misreading them
- Restarting all of my VMs multiple times
- Ensured that I've reassigned the appropriate virtual network device to the public servers in Proxmox
- Various tweaks in my network controller software (Omada) to see if that has anything to do with it

I'm sure there must be something I'm just missing, and it may not be in OPNsense itself, but it seemed the best place to start (and I know there's a lot of this community familiar with running it on Proxmox as well).  I've searched and read through various postings, and none quite line up with what I'm trying to do.

Any assistance would be greatly appreciated, and happy to provide any other details about my configuration that might help.

[To preface:]

To preface:
I am new to both OPNSense and Proxmox, just recently switching my Omada setup to use OPNsense for routing/firewall after being frustrated with the lack of features and capabilities in Omada.  I am approaching this from the perspective that the most likely problem is my lack of knowledge.

The problem:
I have a 1 Gbps symmetrical fiber connection through AT&T, but I'm only able to receive roughly half of that bandwidth down, and about 350 Mbps up using the community speedtest plugin directly from the OPNSense instance (less than that from connected devices).  I had no issues getting full bandwidth (at least 900/900) on both the Omada router, or my previous Asus router setup. 

Things I've tried:
- I have seen all the posts (like this one) about needing to assign a parent interface to the WAN, but I don't think that applies to me because my WAN interface is just using the actual virtual network adapter passed from Proxmox rather than some vlan off of it.
- I've seen a few posts about various tunable options  (hw.ibrs_disable and vm.pmap.pti) and tried those to no noticeable effect. 
- I've tried various CPU virtualization options in Proxmox (host, multiple cores of QEMU, etc.), but CPU usage never really gets much about 50% when running a speedtest, and generally is in the single digits.
- All the "hardware offload" options are disabled as suggested in several older posts.

My setup: 
- HUNSN NRJ02 Mini Firewall PC (Intel Celeron N5105, 4x Intel 2.5G LAN), 32 GB RAM, 500 GB NVMe SSD.
- Proxmox 7.2-7, with OPNSense as the only VM currently.  I've assigned it the host CPU and 8GB RAM at the moment.
- OPNSense is at the latest version, 22.7, but same thing was occurring on 22.1 before upgrading on the off chance it might help this.

Wrapping up:
What else should I try at this point?  I'm really enjoying learning OPNSense (and Proxmox) and for the most part things work "well enough" but I hate to think that I'm basically leaving half or more of my bandwidth unused.  I thought for sure this little firewall PC would have more than enough horsepower for my fairly simple home setup, and allow me to run another server VM or two as well, but was I just wrong to think that?

Any help or guidance from the experts here would certainly be appreciated, and I'll be glad to provide any more insight I can to help solve this.