Over the course of four weeks and endless hours I have tried to get SDN VLANs going; to no avail.
Proxmox 9.09. OPNsense 25.7.5
The following current config without any VLAN-ing is working.
On Proxmox I have:
- vmbr0 = LAN (NIC) 192.168.2.0/24
- vmbr1 = WAN (USB->Ethernet) 192.168.1.118 gw 192.168.1.1 (to get Internet connectivity for updates, etc.)
- vlanzone (ZONE VLAN)
- vnetlan in vlanzone (no subnets) VID 20
- OPNsense VM with
- virtio net0 vbmbr0
- virtio net1 vbmbr1
On OPNsense I have:
- vlan01 for LAN (not assigned)
- LAN vtnet0 VID 20
- WAN vtnet1
Managed switch: FS.com S3700-24T4F
- port 1 default, no VLAN tag = Proxmox
- port 2 default, no VLAN tag = PC 192.168.2.100
As mentioned, the above config works.
Now, this is what I do to get VLANs configured.
Proxmox:
- change net0 from vmbr0 to vnetlan (if I do this first I loose access to Proxmox)
OPNsense:
- change vtnet0 to vlan01
Managed switch: FS.com S3700-24T4F
- ports 13-18 on VLAN 20 access
- port 23 trunk
After these changes:
- I cannot ping anything
- cannot reach any console; except direct connect to Proxmox machine
Since I can't reach OPNsense, I have to get Proxmox back to get to the OPNsense VM shell and revert the interface assignments.
I have done this now too many times to count, and can for the life of me not figure out what I am doing wrong.
I have watched endless YouTubes, but cannot find SDN Proxmox OPNsense VLAN videos; yes, some of the combinations but not all of them.
I am now at a point where I need help with this config. If there is a means to pay someone knowledgeable, I will.
From what I get is that your LAN NIC is attached to one of the ports 13-18 on your switch?
In that case, if you change the vtnet0 on OpnSense to VLAN 20, what you get is a VLAN 20 in a VLAN 20 (QinQ), which certainly will not work.
You should connect the LAN NIC to port 23 (trunked) when you use LAN as VLAN 20 on OpnSense.
W/r to Proxmox, you would then have to use VLAN 20 there, too. It can be done like so:
auto eth0
iface eth0 inet manual
auto vmbr0
iface vmbr0 inet static
bridge-ports eth0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
bridge-mcsnoop 0
#Network bridge
auto vmbr0.20
iface vmbr0.20 inet static
address 192.168.2.5/24
gateway 192.168.2.1
dns-nameservers 192.168.2.1
#Proxmox LAN interface
Thank you... yes, I connected the NIC to the trunk port. (Apologies for not being specific enough).
How do people set this up without loosing connection to Proxmox and OPNsense?
I have reverted back to what I had working, though without VLANs.
I probably should have attached my switch config too... (now attached).
In any case I will try your config. I was close... :) but missed the VLAN lines.
auto vmbr0
iface vmbr0 inet static
bridge-ports eno1
bridge-stp off
bridge-fd 0
#LAN
auto vmbr0.20
iface vmbr0.20 inet static
address 192.168.2.2/24
netmask 255.255.255.0
gateway 192.168.2.1
#VLAN20
Is there anything else I can post to provide more clarity.?
There are some things missing:
1. You do not specify VLAN awareness and which VLANs can be used, so your OpnSense cannot access VLAN 20.
2. Use a netmask or /24, not both.
3. Other things are missing, like bridge-mcsnoop, which is also in this guide: https://forum.opnsense.org/index.php?topic=44159.0
Details matter.
Yes to all... this is the problem when you work with newbies; so much to get wrong.
I took a step back and set-up a test environment, simplified to the bare bones.
First test and validate switch config.
1. 4 access ports, 1. Proxmox (no OPNsense), 2 clients --> all pings in any direction
2. change Proxmox port to access and added this config --> all pings in any direction
Though the config seems excessive?!
auto lo
iface lo inet loopback
auto eno1
iface eno1 inet manual
iface enx0050b61f9434 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.2.2/24
gateway 192.168.2.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
#LAN bridge (untagged, native VLAN)
auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
#WAN bridge
auto vmbr10
iface vmbr10 inet manual
bridge-ports eno1.10
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 10
#LAN VLAN 10 (tagged)
auto eno1.10
iface eno1.10 inet manual
vlan-raw-device eno1
#VLAN 10 subinterface
source /etc/network/interfaces.d/*
which evolved to:
auto lo
iface lo inet loopback
auto eno1
iface eno1 inet manual
auto enx0050b61f9434
iface enx0050b61f9434 inet manual
#WAN (no VLAN)
auto vmbr0
iface vmbr0 inet static
address 192.168.2.2/24
gateway 192.168.2.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 10,30,40,50
#LAN bridge (untagged + VLANs 20,30,40,50)
auto vmbr1
iface vmbr1 inet static
address 192.168.1.2/24
bridge-ports enx0050b61f9434
bridge-stp off
bridge-fd 0
#WAN bridge (no VLAN)
source /etc/network/interfaces.d/*
... after I created VNets:
cat /etc/pve/sdn/vnets.cfg
vnet: vnetlan
zone vlanzone
alias VNet LAN
tag 10
vnet: vnetdmz
zone vlanzone
alias VNet DMZ
tag 30
vnet: vnetiot
zone vlanzone
alias VNet IoT
tag 40
vnet: vnetgst
zone vlanzone
alias VNet Guest
tag 50
... with this zone:
cat /etc/pve/sdn/zones.cfg
vlan: vlanzone
bridge vmbr0
ipam pve
Now the questions:
1. is this configuration suitable for installing OPNsense in a VM on this Proxmox machine?
2. What I mean, other than having to configure OPNsense, there is no further config on Proxmox required?! Correct?
3. When going through the OPNsense setup in the CLI is it correct to select the VNets and not vmbrs (except for WAN) as interface for the LAN, DMZ, IOT, Guest networks?
4. Anything else I need to do to not log me out of Proxmox or OPNsense? :)
IDK anything about SDN in Proxmox, sorry. Never saw the need for that.
All I can tell you is that with your previous setup, where you had your LAN on VLAN 20 access ports and the Proxmox host as trunk, you can do it with the setup I posted, such that the Proxmox host IP map also to the LAN on VLAN 20.
You can then configure the vtnet0 on OpnSense to vmbr0 and set up the LAN on VLAN 20 as well to access all of that LAN and deliver DHCP and DNS services to it. vtnet1 would then be exclusively used for WAN.