VLAN+dhcp won't work

[Dragonfly]

Hello!

Recently I've decided to buy a mini PC to use as a router, with OPNsense on it. It's a Qotom with 6 i211 Intel NICs. Setting up routing, bridging, NAT and DHCP was a breeze. I'll try to explain what I have, what I want and what I tried:

What I have
- cable modem in bridge mode
- OPNsense box with 1 WAN interface in DHCP
- bridge0 created out of all interfaces (including WAN, because the modem has a web interface to monitor Docsis signal values) with subnet 192.168.0.1/24 and DHCP enabled. All LAN interfaces are set to enabled but have no configuration. In the firewall settings, I've allowed all in and out traffic for now.
- 2 ports are connected to two different Ubiquity EdgeSwitch 10X switches.
- My Linux NAS is connected to one switch and has an untagged default VLAN with a static IP in 192.168.0.1/24.
- In addition, it has a VLAN interface, let's say eth0.10 with 802.1q tag 10. The EdgeSwitch is set up to accept tagged VLAN 10 frames to and from this port. It has a static ip 10.0.10.2/24 on this interface
- The other EdgeSwitch has an almost similar Linux HTPC connected with a static IP in 192.168.0.1/24 and a VLAN 10 interface with a static ip 10.0.10.3/24.

I can ping and connect to various TCP ports from 10.0.10.3 and 10.0.10.2, so it works. I also set up a tagged VLAN 10 interface on my Macbook and this also works (wired). This also already worked before I bought the OPNsense box and everything was still connected to my ISP's modemrouter in router mode. This situation still works with my OPNsense box.

What I want
- A DHCP server for VLAN 10 for 10.0.10.0/24

What I tried
- Create a VLAN with tag 10 and parent interface bridge0
- Assign the new interface and set a static ipv4 of 10.0.10.1/24
- Apply the interface changes
- Enable the DHCPd4 service for the interface vlan_10 with a range of 10.0.10.100-10.0.10.200
- Apply the dhcpd changes
- Allow all traffic in and out the vlan_10 interface in the firewall.
- Apply firewall changes

Yet when execute sudo dhclient -v eth0.10 on either box, it won't get any kind of response. It just keeps doing a DHCPDISCOVER on an increasing interval until it gives up.

Does anyone have any idea what I've omitted/forgotten/misconfigured? Any help would be appreciated!

[marjohn56]

Do you want to post a network diagram of what you are trying to do. Generally bridging is the last resort when you don't have switches.

[Dragonfly]

Sure, but it's quite simple.



The dotted lines are tagged VLAN connections. The solid lines are physical cat6 cables. I know bridging happens in software and a switch can dedicate itself to switching at line speed, but hey, my OPNsense box is a relatively new i5, so it should be able to both route and bridge at Gigabit speeds (and it does).

What I'm trying to do is to get 10.0.10.2 and 10.0.10.3 replaced by a 10.0.10.x ip through DHCP. Because whilst in the current situation with tagged VLAN ports and static addresses, it works perfectly well, this won't be the case when I buy a VLAN aware UniFi switch which also connects IoT devices. This PoC is basically in preparation for my wireless also going VLAN.

[marjohn56]

OK, I see. I do it in a different way.


It's much simpler just to put an un-managed or managed switch directly after the Qotom in the 'Fuse Box Closet', then you just 'trunk' everything out of a single port on the Qotom to that switch and then onto the other switches; you can then use a separate port on the Qotom for the cable modem management. Doing it that way reduces the CPU load on the Qotom,


I also run multiple VLANs and have a connection to the modem for monitoring purposes. I've never tried using bridge mode with VLANs, but I'll run it up on my test Qotom and see what gives, but  I would still suggest doing it the way I do it.

[Dragonfly]

I've logged in through SSH to inspect see various odd things.

First of all, I don't appear to have a /etc/dhcpd.conf, even though the process specifies that config file:

/usr/local/sbin/dhcpd -user dhcpd -group dhcpd -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid bridge0 bridge0_vlan10

In addition, the tag on my VLAN interface doesn't seem to have been set properly?

bridge0_vlan10: flags=8003<UP,BROADCAST,MULTICAST> metric 0 mtu 1500
   ether 00:00:00:00:00:00
   inet6 fe80::4262:31ff:fe0b:7a60%bridge0_vlan10 prefixlen 64 tentative scopeid 0xc
   inet 10.0.10.1 netmask 0xffffff00 broadcast 10.0.10.255
   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
   vlan: 0 vlanpcp: 0 parent interface: <none>
   groups: vlan

[Dragonfly]

OK, I see. I do it in a different way.


It's much simpler just to put an un-managed or managed switch directly after the Qotom in the 'Fuse Box Closet', then you just 'trunk' everything out of a single port on the Qotom to that switch and then onto the other switches; you can then use a separate port on the Qotom for the cable modem management. Doing it that way reduces the CPU load on the Qotom,


I also run multiple VLANs and have a connection to the modem for monitoring purposes. I've never tried using bridge mode with VLANs, but I'll run it up on my test Qotom and see what gives, but  I would still suggest doing it the way I do it.

Thanks for you quick replies, but I don't think that that will solve my problem. I've basically already tried it by just creating the VLAN interface with igb1 as parent interface rather than bridge0. But it still won't answer DHCPcd/dhclient request.

I do see via SSH that it does have a vlan tag now though:

igb1_vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   ether 40:62:31:0b:7a:61
   inet6 fe80::4262:31ff:fe0b:7a61%igb1_vlan10 prefixlen 64 scopeid 0xc
   inet 10.0.10.1 netmask 0xffffff00 broadcast 10.0.10.255
   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
   vlan: 10 vlanpcp: 0 parent interface: igb1
   groups: vlan

[marjohn56]

OK, let's go back to basics. Lets do a checklist:


1. Interfaces->Other Types->VLAN: Add the VLAN and assign the interface, eg igb2
2. Interfaces->Assignments->Add the new VLAN, OPTx or whatever
3. Select the new interface and configure, we'll rename it as VLAN_IOT
So it needs to be enabled. Static address set for IPv4, If you use IPv6 then that;s a seperate ballgame and we'll leave that for now.
Make sure that the static IPv4 address has the correct mask, it defaults to 32, and obviously needs to be something else, 24 is normal.
That's it on the interfaces.
Goto Services->DHCPv4 and you should have the VLAN interface listed, select it.
Make sure it's enabled, set the range from/to. Save and apply...


That's it.


If you still have a problem, disable the dhcpv4 on any other interfaces and see if its another interfaces that's causing dhcpv4 to barf.

[marjohn56]

Just completed testing here - working fine.


Did you select one the physical ports as the parent for the VLANs or the bridge interface?


The VLANs need to be attached to the physical interface, not the bridge interface.

[Dragonfly]

Just completed testing here - working fine.


Did you select one the physical ports as the parent for the VLANs or the bridge interface?


The VLANs need to be attached to the physical interface, not the bridge interface.

I checked the entire list. In addition I made sure all physical and VLAN interfaces allow all incoming and outgoing traffic. All physical interfaces have been configured without ipv4 or ipv6 configuration.

The parent interface for the VLAN is igb1, so a physical NIC.

[russella]

I had a problem with getting a DHCP lease on a VLAN and it was related to the Intrusion Detection service. With the Intrusion Detection service enabled devices connecting on a network associated with a VLAN couldn't get a DHCP lease. With the Intrusion Detection service disabled they could. I didn't want to disable the Intrusion Detection service so I eventually found that I could leave it enabled if I disabled VLAN Hardware Filtering (Interfaces->Settings->VLAN Hardware Filtering=Disable VLAN Hardware Filtering).

[marjohn56]

I'll check this again. I used my PC second WAN port as a test connected directly to one of the bridge interfaces and just changed the VLAN setting on the PC port, it got an address on bot the VLANs I'd created.


let me check again, it may be something to do with the tunables..

[marjohn56]

No, I take it back. It won't work as the VLAN is attached to the physical interface, or rather it will only work on the physical interface it's attached to, and you cannot attach a VLAN to the bridge so that's a no go; when I tested I only tested the LAN bridge was working and the VLAN was working on the port I was connected to; I wrongly assumed that the VLAN was working on all the ports, so sorry about that.


I'm afraid the solution is going to have to be a switch after the Qotom as I originally thought or you have separate VLANs attached to each physical port and just have different address ranges for you Lounge and Living Room, you can allow traffic to flow between by just adding the appropriate fw rules.

[Dragonfly]

I had a problem with getting a DHCP lease on a VLAN and it was related to the Intrusion Detection service. With the Intrusion Detection service enabled devices connecting on a network associated with a VLAN couldn't get a DHCP lease. With the Intrusion Detection service disabled they could. I didn't want to disable the Intrusion Detection service so I eventually found that I could leave it enabled if I disabled VLAN Hardware Filtering (Interfaces->Settings->VLAN Hardware Filtering=Disable VLAN Hardware Filtering).

Whoah! This solved the problem for me! Thanks! I never would have found this out by myself!

I don't have IDS enabled (yet), but disabling VLAN HW Filtering solved my problem.

I do still find it unfortunate that I can create a VLAN interface with a bridge interface as parent. Well I can, but then it doesn't get a 802.11q tag so it never works. I don't see a reason why this shouldn't technically be possible so I see this as a OPNsense shortcoming tbh.

But creating VLANs for every physical interface and then bridging said interfaces into a vlan10_bridge works fine.  It's just a bit more work and fairly clunky to manage.

[mb]

Hi dragonly,

I'm trying to understand whether your problem is related to netmap(4) and is a discussion for this thread: https://forum.opnsense.org/index.php?topic=17363.30

When you say dhcp does not work with vlan interface which is a child of a bridge interface (e.g. bridge0_vlan0), do you have Suricata in IPS mode enabled on any of the interfaces? (i.e. bridge member interfaces, bridge itself or the vlan interface?)

[Dragonfly]

No, IDS/IPS has been fully disabled. Basically it's a clean setup with only a WAN, bridge and a VLAN configured.