Bridged VLAN

[simonmb]

I'm migrating from EdgeOS to OPNsense and struggling to create a vlan that spans multiple physical interfaces.

Ultimately I'm trying to create 5 subnets
MGMT   .11
HOME   .12
IOT    .13
CCTV   .14
GUEST  .19

Assigned to the following physical interfaces
IGC0                     -> WAN
IGC1   .11               -> PC
IGC2   .11,.12,.13,.19   -> Switch & AP
IGC3   .14               -> NVR


I currently have the default config of WAN and LAN (192.168.1) and I'm trying to create the MGMT vlan across IGC2 and IGC3 (for testing) using this post as reference https://forum.opnsense.org/index.php?topic=42084.msg207401#msg207401

I created a vlan on each port
Device        Parent   Tag
vlan0.2.11    igc2     11
vlan0.3.11    igc3     11

I assigned them an interface
Device       Interface
vlan0.2.11   vlan_MGMT_02
vlan0.3.11   vlan_MGMT_03

I added both to a bridge
Interface   Members
bridge0     vlan_MGMT_02,vlan_MGMT_03

Assigned it an interface
Interface   Device
MGMT        bridge0

Enabled the MGMT interface and assigned it a static IP of 192.168.11.1
Enabled the DHCP Server on the MGMT Interface

I don't believe this matters at this point but I also changed
System ‣ Settings ‣ Tunables and change
net.link.bridge.pfil_member to 0
net.link.bridge.pfil_bridge to 1

I believe the 'Automatically generated rules' allow access to the DHCP server on the MGMT interface (I have tried creating an allow all rule) but when I plug my PC into one of the ports and issue ipconfig /release; ipconfig /renew the process times out with the following error:

An error occurred while renewing interface Ethernet : unable to contact your DHCP server. Request has timed out.

Is this possible, and if so where am I going wrong?

Thanks in advance

[Patrick M. Hausen]

I assigned them an interface
Device       Interface
vlan0.2.11   vlan_MGMT_02
vlan0.3.11   vlan_MGMT_03

Not necessary.

Assigned it an interface
Interface   Device
MGMT        bridge0

Correct.

I don't believe this matters at this point but I also changed
System ‣ Settings ‣ Tunables and change
net.link.bridge.pfil_member to 0
net.link.bridge.pfil_bridge to 1

Absolutely necessary, but you did it, so also good.

What you now created is two trunk ports that are bridged so you can connect e.g. two switches or APs that also run VLAN 11 tagged. If you connect a PC with an untagged port of course not much will happen.

What is the intended final setup? Connect network devices (tagged) or PCs (untagged)?

[simonmb]

I assigned them an interface
Device      Interface
vlan0.2.11  vlan_MGMT_02
vlan0.3.11  vlan_MGMT_03

Not necessary.

Given a bridge only accepts interfaces as members how would I link them?

I don't believe this matters at this point but I also changed
System ‣ Settings ‣ Tunables and change
net.link.bridge.pfil_member to 0
net.link.bridge.pfil_bridge to 1

Absolutely necessary, but you did it, so also good.

I thought that was "just" about where the firewall rules were applied, which for this test didn't really matter?

What you now created is two trunk ports that are bridged so you can connect e.g. two switches or APs that also run VLAN 11 tagged. If you connect a PC with an untagged port of course not much will happen.

That's what I was thinking, but your reply to this post had me believe it should be possible https://forum.opnsense.org/index.php?msg=138047
I've clearly misinterpreted something.

What is the intended final setup? Connect network devices (tagged) or PCs (untagged)?

Ultimately
IGC0                   -> WAN
IGC1  .11              -> PC
IGC2  .11,.12,.13,.19  -> Switch (vlan aware) & AP
IGC3  .14              -> NVR

but for this test it was
IGC0                   -> WAN
IGC1  .1               -> PC
IGC2  .11              -> PC
IGC3  .11              -> Switch (vlan aware) & AP

[dseven]

What's ".1"?

Are you doing VLAN tagging on the PC? Getting this right can be tricky, especially on Windows.

[simonmb]

What's ".1"?

That's the current subnet (192.168.1). All the others (.11,.12) represent the subnet and VLAN tag.

Are you doing VLAN tagging on the PC? Getting this right can be tricky, especially on Windows.

I'm not.

I'm trying to recreate my EdgeOS setup in OPNsense but it sounds like it's not that simple.

[dseven]

But what interface on OPNsense is ".1" on?

Note that you cannot bridge the untagged VLAN on a NIC and also use tagged VLANs on that NIC at the same time.

[simonmb]

But what interface on OPNsense is ".1" on?

Note that you cannot bridge the untagged VLAN on a NIC and also use tagged VLANs on that NIC at the same time.

During testing IGC1 is assigned the LAN interface (192.168.1) which my PC is connected to.
I'm trying to bridge IGC2 & IGC3 with a MGMT vlan (192.168.11) hoping that if I move my PC from IGC1 to IGC2/IGC3 it'll pick up a 192.168.11 address.

[Patrick M. Hausen]

Sorry, if the bridge only accepts assigned interfaces as members, then yes, you need to assign the VLANs. You run e.g. VLAN 11 tagged on igc2. To connect a PC on igc1 to VLAN 11 you need to bridge the VLAN on igc2 and the untagged igc1.

But why not plug the PC into the switch which is much better at these things?

[dseven]

So the PC connected to IGC1 should work on LAN. Doesn't it?

The problem, however, is that VLAN 11 is tagged on IGC2 and IGC3, so the PC will only be able to use it if it can handle VLAN tagging, or if you connect a managed switch between one of those ports and the PC, and have the switch handle the tagging (as Patrick pointed out before).

[pfry]

[...]
I'm trying to recreate my EdgeOS setup in OPNsense but it sounds like it's not that simple.

Possibly. (I've never looked at Ubiquity.) Looks like you're conflating a switch model for VLANs with OPNsense's router model. On OPNsense, a VLAN is only useful in separating traffic on a single interface. It is not relevant to separating traffic over multiple interfaces.

So let's start from the beginning. I'm going to channel Patrick and ask "Why are you using bridges?" On OPNsense, bridges are useful for extending L3 configuration across interfaces. For instance, I use four bridges on my firewall; this allows me to use only four rulesets and three DHCP subnets for the three "internal" bridges (which each have multiple interfaces assigned). If this is what you're looking for, you're on the right track. Consider your goals, though - a bridge may not be appropriate for your Internet edge, for instance.

I do use VLANs. I (try to) run all of my devices through the firewall (devices connected to a wi-fi AP are a special case). To gain more ports I have a switch which I have configured as a port expander (by assigning each of the first nine interfaces to a unique VLAN and the tenth to all, as a trunk), plugged into an interface on the firewall with appropriate VLANs configured. So, as above, I use this to separate traffic on a single interface.

Example: My "Interfaces: Devices: Bridge" looks like:

Code Select Expand

Interface Members Description Link-local
bridge0 vlan103, vlan104, x550, x710p0, x710p2 Edge Off
bridge1 rt8125, vlan105, vlan107, vlan109 Trust Off
bridge2 vlan106, vlan108 Guest Off
bridge3 vlan1, vlan102 Jail Off

The VLANs are all attached to x710p3. Naturally the main interface is not configured or assigned to a device.

[simonmb]

[...]
I'm trying to recreate my EdgeOS setup in OPNsense but it sounds like it's not that simple.

Possibly. (I've never looked at Ubiquity.) Looks like you're conflating a switch model for VLANs with OPNsense's router model.

I think that's probably the crux of the problem.

So let's start from the beginning. I'm going to channel Patrick and ask "Why are you using bridges?"

I was using a bridge to combine subnets across physical interfaces, but let's start afresh with what I'm currently doing and maybe there's a better way.

I have my router, switch and AP all on 192.168.11.0/24.

I've configured an interface on the router to carry 5 vlans (192.168.11.0/24, .12, .13, .19) to a switch, which an AP (carrying same vlans) is connected.

One port on my router (192.168.14.0/24) is connected to an NVR.

I mostly manage my networking equipment from a computer on 192.168.12.0/24 which is connected to the switch (access controlled by firewall rules).

Occasionly I've connected a laptop directly to the router (the port assigned to 192.168.11.0/24) when I've suspected problems with the switch, or if I've misconfigured access from 192.168.12.0/24, although I can't even remember the last time I did it, but because I had spare ports I thought I'd replicate it.

[simonmb]

So the PC connected to IGC1 should work on LAN. Doesn't it?

Yes the PC is fine on IGC1.

The problem, however, is that VLAN 11 is tagged on IGC2 and IGC3, so the PC will only be able to use it if it can handle VLAN tagging, or if you connect a managed switch between one of those ports and the PC, and have the switch handle the tagging (as Patrick pointed out before).

I think that's what I hadn't appreciated initially as the posts I had seen made no mention of it; I guess because it was obvious/assumed.

Thanks for you help

[simonmb]

But why not plug the PC into the switch which is much better at these things?

It wasn't intended for regular use, I'd previously connected a laptop to igc1 for troubleshooting when I suspected a problem with the switch.

It sounds like my options include:

• Create another subnet on IGC1 with access to manage the router.
• Configure my laptop to accept tagged vlans
• Access the router over the serial interface

Thanks for your help

[pfry]

It wasn't intended for regular use, I'd previously connected a laptop to igc1 for troubleshooting when I suspected a problem with the switch.

It sounds like my options include:

• Create another subnet on IGC1 with access to manage the router.
• Configure my laptop to accept tagged vlans
• Access the router over the serial interface

Thanks for your help

Is having the PC connected directly to igc1 a long-term plan? Your initial troubleshooting effort was good, but under normal operation would you expect any reliability issues from the switch? If both are "yes", just use igc1 directly - no VLAN. Same with igc3, if the NVR is the only connected device. I like to keep some interfaces handy for that sort of thing, as you can see from my bridge list. I expect you'll have fewer VLANs, as you do not appear to be going for the same isolation level.

As for the WAN/Internet interface, you have... well, lots of options, depending on how you want to plug it up.

With a bridged ".11" subnet you could place other devices in the subnet, connected through the firewall, and filter them (e.g. other devices on the ".11" bridge, like your PC on igc1). This has the advantage of allowing you to connect said equipment directly to your router and bypass the firewall if necessary/desired.

On the other hand, with a separate router, you (may) have the option to forego the bridge and assign a ".11" IP directly to igc0. Is that what you were looking at for your first option above, reserving the ".11" subnet for the router link alone? It's a choice. About the only downsides I see to the bridge are the potential for ARP (specifically ARP proxies) or broadcast/multicast issues (a bit ugly - just take them into account when you design your filters). Oh, and differences between filtered (through the firewall) and unfiltered, depending on how individual devices are connected. Others may chime in with things I've missed.

[simonmb]

Is having the PC connected directly to igc1 a long-term plan? Your initial troubleshooting effort was good, but under normal operation would you expect any reliability issues from the switch?

I don't know if you saw my reply to your message but I added a little extra detail:

let's start afresh with what I'm currently doing and maybe there's a better way.

I have my router, switch and AP all on 192.168.11.0/24.

I've configured an interface on the router to carry 5 vlans (192.168.11.0/24, .12, .13, .19) to a switch, which an AP (carrying same vlans) is connected.

One port on my router (192.168.14.0/24) is connected to an NVR.

I mostly manage my networking equipment from a computer on 192.168.12.0/24 which is connected to the switch (access controlled by firewall rules).

Occasionly I've connected a laptop directly to the router (the port assigned to 192.168.11.0/24) when I've suspected problems with the switch, or if I've misconfigured access from 192.168.12.0/24, although I can't even remember the last time I did it, but because I had spare ports I thought I'd replicate it.

As for the WAN/Internet interface, you have... well, lots of options, depending on how you want to plug it up.

With a bridged ".11" subnet you could place other devices in the subnet, connected through the firewall, and filter them (e.g. other devices on the ".11" bridge, like your PC on igc1). This has the advantage of allowing you to connect said equipment directly to your router and bypass the firewall if necessary/desired.

To be honest I want to keep things as simple as possible because my knowledge in this area is limited. I want the vlans to do the heavy lifting for network segregation, and a handful of firewall rules to allow access from HOME (.12) to CCTV (.14) and mDNS and SSDP between HOME (.12) and IOT (.13) for audio streamers.