OPNSense VM does not respect vlan TAG from hypervisor bridge

[JL]

Hey, please think with me for a moment.
 
Using 24.10 in a VM hosted on a Linux VM server. The OPNSense VM is connected to a Linux bridge which simply passes all (tagged) vlan from the interface connected to the switch.

Observation: vlan traffic is seen on the physical interface and bridge with the vlan tag present, the switch only offeres tagged vlan
Problem: inside the VM though, the traffic seems untagged since it is not observed on the vlan0.401 interface for example
Validation: when connecting another VM to vlan0.401 the communication works well
Question: how to fix that the vlan tag from the hypervisor bridge is passed to the opnsense vlan interface from the parent

Linux Bridge config looks like below, the opnsense vlan is attached to the parent which has assinged the bridge interface on the hypervisor host.
---
auto LIF
iface lif inet static
        bridge-ports eth1
        bridge-stp off
        bridge-vlan-aware yes
        bridge-vids 401 402 901 1500
        
For one other interface the bridge has a 1:1 mapping like, this works well since the vlan is not "inside" the VM

auto DIF
iface dif inet static
        bridge-ports eths3.700
        bridge-stp off
        bridge-vlan-aware yes
        bridge-vids 700

I'd prefer to 'pass through' the interface to the VM but this can only be done over a bridge, leading to the current problem situation.

Br,

JL

[Patrick M. Hausen]

You can use PCIe passthrough of the network interface and let the OPNsense handle all of the VLAN stuff. No idea about your specific setup, I don't run OPNsense that way, sorry.

[JL]

You can use PCIe passthrough of the network interface and let the OPNsense handle all of the VLAN stuff. No idea about your specific setup, I don't run OPNsense that way, sorry.

Thanks for the reply. I'm trying that now, escaped me to try this.

I had something like the my setup described working in the past but forgot to document it properly.

The issue is not unique apparently and a repeat iritation with user using opnsense in a VM with forced use of a bridge.

edit /

i did find this is likely a know issue with Linux
https://wiki.debian.org/NetworkConfiguration#Bridging_without_Switching

another article speaks about how ipv6 can cause a bridge to fail
more here https://wiki.linuxfoundation.org/networking/bridge

https://wiki.linuxfoundation.org/networking/bridge#no_traffic_gets_trough_except_arp_and_stp

[JL]

SOLVED, thanks to the articles below,

Main reason assumed is MAC address are the same for all vlan (obviously) as the Debian page documents

auto LIF
iface lif inet manual
        bridge-ports eth0
        bridge-setageing 0
        bridge-stp off
        bridge-fd 1
        bridge-vlan-aware yes
        bridge-vids 401 402 901 1500
        mtu 1422


I'm not certain if the MTU reduction is required, should not matter much.

on the switch the port is now again set to General - accept tagged only

inside OPNSense the vlan interface has the MAC of the parent set, no other modification was made to the vlan interface
at this point i don't think that really matters

I also set this tunable to 1 : Select the tunable net.link.bridge.pfil_bridge and set the value to 1

https://docs.opnsense.org/manual/how-tos/lan_bridge.html#step-six

[Patrick M. Hausen]

Do you have a bridge interface in OPNsense? What for? I thought you want to connect a VLAN interface or actually some of them through a Linux vmbr, so probably a virtio interface (vtnetX) in OPNsense?

[meyergru]

I have a similar setup running under Proxmox which works fine.

FWIW, the bridge setup there is:

Code Select Expand

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eth1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        bridge-mcsnoop 0

And Patrick is right: What is the OpnSense bridge for? You would need a parent interface that connects to the Linux vmbr0 and then create VLAN interfaces from that as needed.

[EricPerl]

The Proxmox bridge used for the LAN side of my OPN VM is not even LAN aware.
Straight vtnet to vmbr mapping and all VLANs have vtnet as parent. I used to have the exact same config over PCIe passthrough.
vtnet is not assigned to any interface directly because I no longer have untagged traffic. No issues.

It's my understanding that VLAN awareness comes into play if you want to constrain a VM to a specific VLAN.

[meyergru]

Correct, the VLAN awareness is not strictly needed for OpnSense itself, but for any other VM that is to be connected to one of the VLANs. This potentially includes the Proxmox host itself.

Also, you can attach physical interfaces to a specific VLAN on Proxmox:

Code Select Expand

auto eth1
iface eth1 inet manual
#10 Gbit/s Trunk

auto eth2
iface eth2 inet manual
        bridge-access 10
#10 Gbit/s NAS Direct Access LAN only

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eth1 eth2
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        bridge-mcsnoop 0
#Network bridge