VLAN devices are on LAN IPs

[dseven]

So... damned if you do, damned if you don't 🤣🤣

[nero355]

So... damned if you do, damned if you don't 🤣🤣

Well... you shouldn't... so... you have got a choice... sort of... LOL! ^_^

[tonys]

I went back to my original configuration as posted in the beginning of this thread (listed from the SSH login to OPNSense):

*** OPNsense.home.lan: OPNsense 26.7.1_1 (amd64) ***

 DMZ (igc2)      -> v4: 192.168.30.1/24
 Guest (vlan01)  -> v4: 192.168.20.1/24
 IoT (vlan02)    -> v4: 192.168.40.1/24
 LAN (igc0)      -> v4: 192.168.1.1/24
 WAN (igc1)      -> v4/DHCP4: x.x.x.x

From one of my LAN devices (192.168.1.63), I attempted pings to the Guest (192.168.20.1) and IoT (192.168.40.1) vlans and there is no connectivity as expected. This means there are no leaks that could lead to bleeding FROM the LAN TO the Guest or IoT networks. Unfortunately, I still have no devices connecting on either vlan - all devices are getting LAN IPs. I don't see how to check for bleeding from the Guest or IOT vlans to the LAN since I can't get any device onto either vlan.

This is bad - the Guest and IoT devices have complete access to my LAN. How do I check for leakage if I can't get any devices to connected to either VLAN?

[dseven]

At the start, you had a bridge. Is that bridge now completely gone? Updated ifconfig output might be helpful.

Otherwise I'd tend to suspect that your WiFi is misconfigured...

[viragomann]

I went back to my original configuration as posted in the beginning of this thread (listed from the SSH login to OPNSense)

So I could repeat my first post as well. ;-)

From one of my LAN devices (192.168.1.63), I attempted pings to the Guest (192.168.20.1) and IoT (192.168.40.1) vlans and there is no connectivity as expected. This means there are no leaks that could lead to bleeding FROM the LAN TO the Guest or IoT networks.

No. This just means, that there is no layer 3 leak, but VLANs must separate network segments at layer 2.

Explanation: When the (LAN) device tries to access an IP in another network segment it doesn't do ARP resolution, but sends to packets directly to the gateway, which is the firewall. If access to the other subnet isn't permitted here, the access fails of course.

Unfortunately, I still have no devices connecting on either vlan

However, to get an IP from a DHCP server, a device sends the request to the broadcast. But since it doesn't have a network setting at this point, it sends the request to the overall broadcast address,  which is 255.255.255.255, and this goes across layer 3 boundaries. So it also reaches a DHCP in the obviously other subnet.

So your network setup might leak at layer 2.

[dseven]

However, to get an IP from a DHCP server, a device sends the request to the broadcast. But since it doesn't have a network setting at this point, it sends the request to the overall broadcast address,  which is 255.255.255.255, and this goes across layer 3 boundaries. So it also reaches a DHCP in the obviously other subnet.

That's not really what happens. Broadcasts never "reach" between subnets. In the case of a broadcast DHCP request, there is no subnet yet. The DHCP packet gets transmitted in a broadcast ethernet frame (destination ff:ff:ff:ff:ff:ff). Any DHCP server in that layer 2 domain can reply to that request. The ethernet frame containing the request can (and should, in this case) include a VLAN tag. When OPNsense receives such tagged frames on igc0, if there is a correctly-configured VLAN subinterface for that VLAN ID, that interface will handle the frame. If there's a DHCP service listening on that interface, it should respond accordingly. Otherwise it should be ignored.

Here's an example of a DHCP request on VLAN 5 as seen by tcpdump:

Code Select Expand

root@opnsense:~ # tcpdump -nnei igc0 vlan and port 67 or port 68
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igc0, link-type EN10MB (Ethernet), snapshot length 262144 bytes

08:58:48.234885 52:3f:a3:79:d1:2c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 354: vlan 5, p 0, ethertype IPv4 (0x0800), 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:3f:a3:79:d1:2c, length 308
08:58:48.235421 00:d0:b4:01:a5:a1 > 52:3f:a3:79:d1:2c, ethertype 802.1Q (0x8100), length 350: vlan 5, p 0, ethertype IPv4 (0x0800), 10.1.5.1.67 > 10.1.5.201.68: BOOTP/DHCP, Reply, length 304
...

... and an example on VLAN 11, for which no VLAN interface exists:

Code Select Expand

root@opnsense:~ # tcpdump -nnei igc0 vlan and port 67 or port 68
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igc0, link-type EN10MB (Ethernet), snapshot length 262144 bytes

09:05:50.914317 66:78:39:f8:6b:89 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 350: vlan 11, p 0, ethertype IPv4 (0x0800), 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 66:78:39:f8:6b:89, length 304
09:05:51.970468 66:78:39:f8:6b:89 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 350: vlan 11, p 0, ethertype IPv4 (0x0800), 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 66:78:39:f8:6b:89, length 304
09:05:54.123988 66:78:39:f8:6b:89 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 350: vlan 11, p 0, ethertype IPv4 (0x0800), 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 66:78:39:f8:6b:89, length 304


[viragomann]

That's not really what happens. Broadcasts never "reach" between subnets. In the case of a broadcast DHCP request, there is no subnet yet. The DHCP packet gets transmitted in a broadcast ethernet frame (destination ff:ff:ff:ff:ff:ff). Any DHCP server in that layer 2 domain can reply to that request.

So what is different here to that I wrote??
ff:ff:ff:ff:ff:ff = 255.255.255.255
^^

The ethernet frame containing the request can (and should, in this case) include a VLAN tag.

And this is the part, which might not work here.

Yes, you're right, the broadcast shouldn't leave its VLAN. But the VLAN separation doesn't seam to work at layer 2.

[dseven]

That's not really what happens. Broadcasts never "reach" between subnets. In the case of a broadcast DHCP request, there is no subnet yet. The DHCP packet gets transmitted in a broadcast ethernet frame (destination ff:ff:ff:ff:ff:ff). Any DHCP server in that layer 2 domain can reply to that request.

So what is different here to that I wrote??

You wrote "... this goes across layer 3 boundaries. So it also reaches a DHCP in the obviously other subnet.", which makes it sound like the DHCP request gets routed at layer 3. It doesn't.

The ethernet frame containing the request can (and should, in this case) include a VLAN tag.

And this is the part, which might not work here.

Yes, you're right, the broadcast shouldn't leave its VLAN. But the VLAN separation doesn't seam to work at layer 2.

What are you based that assessment on? It works for me...

[viragomann]

You wrote "... this goes across layer 3 boundaries.

That's correct. ff:ff:ff:ff:ff:ff, the MAC broadcard address, goes to any device in the local layer 2.
If the VLAN segmentation doesn't work, it goes to any devices in other alleged VLANs as well. And this is, what we're talking here about, the basic issue of this thread, we are trying to resolve.

What are you based that assessment on? It works for me...

The issue of the TO.
It works in my setup as well, but I don't complain.
^^

[dseven]

I don't know what a "TO" is, but I have not yet seen any proof that a DHCP request from a tagged ethernet frame is getting a response from the DHCP service on the base ("untagged") interface. I suspect that the request frame is not actually arriving tagged, which is why I suspect WiFi misconfiguration. I'd suggest that the OP (original poster) double-check that first, then maybe try tcpdump like I showed...

Also I wanted to confirm that the bridge is completely out of the picture, because it could be doing something silly like grabbing the tagged frame, stripping the tag, and dumping it onto its member interfaces.

[viragomann]

I don't know what a "TO" is

The thread opener.

but I have not yet seen any proof that a DHCP request from a tagged ethernet frame is getting a response from the DHCP service on the base ("untagged") interface

Again, I didn't ever complain that this is possible in a correctly configured network with subnets. But we are trying to solve an issue here and I tried to give a possible explanation for it.
Man!!!

[Bob.Dig]

I bet it is layer 8.

[tonys]

At the start, you had a bridge. Is that bridge now completely gone? Updated ifconfig output might be helpful.

Otherwise I'd tend to suspect that your WiFi is misconfigured...

@viragomann... here is the updated config and yes, the bridge is completely gone and igc3 is disconnected. Sorry about the confusion.

Notes:

- VLAN02 and VLAN03 are unresponsive from anywhere other than OPNSense itself. I can ping 192.168.20.1 and 192.168.40.1 but only when I'm logged directly into the OPNSense shell via SSH. I don't know what layer these pings are operating from.

- The VLAN networks are enabled in UnifiOS and labeled correctly as Guest and IoT with the appropriate tags (20 and 40 respectively). I highly doubt there is a misconfiguration in Unifi OS as it's the same as it always has been and worked just fine under all OPNSense releases up to and including 21.x. The devices listed in Unifi OS are shown on the correct networks (LAN, Guest, and IoT) but they all have LAN IPs. All of my problems started after upgrading to 26.7.x.

- OPNSense's ARP table clearly shows the issue - both VLANs are shown with IP's of 192.168.20.1 and 192.168.40.1 but none of the devices can reach them. Instead, OPNSense incorrectly dumps them into LAN IPs.

igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: LAN (lan)
    inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (2500Base-T <full-duplex>)
    status: active

igc1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: WAN (wan)
    inet xxx.xxx.xxx.xxx netmask 0xfffff800 broadcast 255.255.255.255
    inet6 x cac%igc1 prefixlen 64 scopeid 0x2
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active

igc2: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: DMZ (opt1)
    inet 192.168.30.1 netmask 0xffffff00 broadcast 192.168.30.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active

igc3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    media: Ethernet autoselect
    status: no carrier

lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
    inet 127.0.0.1 netmask 0xff000000
    groups: lo

enc0: flags=0 metric 0 mtu 1536
    options=0
    groups: enc

vlan00: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    groups: vlan
    vlan: 10 vlanproto: 802.1q vlanpcp: 0 parent interface: igc0
    media: Ethernet autoselect (2500Base-T <full-duplex>)
    status: active

vlan01: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: Guest (opt2)
    inet 192.168.20.1 netmask 0xffffff00 broadcast 192.168.20.255
    groups: vlan
    vlan: 20 vlanproto: 802.1q vlanpcp: 0 parent interface: igc0
    media: Ethernet autoselect (2500Base-T <full-duplex>)
    status: active

vlan02: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: IoT (opt3)
    inet 192.168.40.1 netmask 0xffffff00 broadcast 192.168.40.255
    groups: vlan
    vlan: 40 vlanproto: 802.1q vlanpcp: 0 parent interface: igc0
    media: Ethernet autoselect (2500Base-T <full-duplex>)
    status: active