How do IPv6 Router Advertisements get across VLANs?

[barney]

I'm trying to add some matter devices to my setup with devices on an IoT VLAN (40) and the OpenHab controller on a separate VLAN (20), but a router advertisement broadcast on VLAN 40 does not appear to make it to VLAN 20.

I'm trying to understand which piece of the IPv6 puzzle is responsible for passing this RA between VLANs, and what I need to do to get it working.

Some more info... I have the following set up:

OPNsense on a PC Engines apu4d4
Ubiquiti Pro Max 16 PoE switch
R-Pi (Raspbian Trixie) running OpenHab on VLAN 20
R-Pi (Raspbian Trixie) for testing on VLAN 40
IKEA Dirigera on VLAN 40

• VLANs are defined in OPNsense and mirrored with the same VLAN tags in the switch configuration.
• All the devices are hard wired (no WiFi) to the switch, with the OPNsense port in trunk mode assigned to the default Ubiquiti VLAN and the other ports in access mode assigned to their relative VLANs (20 for OpenHab, 40 for Dirigera and test R-Pi).
• When the Dirigera sends out an RA it turns up on the test R-Pi on the same VLAN but not on the OpenHab R-Pi on a different VLAN.
• My ISP does not have IPv6 so I have created a static /56 ULA and assigned :20/64 and :40/64 subnets to the VLANs. WAN, LAN and other VLANs are set to IPv6: None.
• I've enabled Services > Router Advertisements on both VLAN 20 and 40 interfaces, using Unmanaged mode.
• I'm running a FreeBSD port of dennypage/mdns-bridge on the server, although I can achieve the same results with the IoT config available on the Ubiquiti switch.
• I have no DHCPv6 running - seems like SLAAC should be sufficient for what I'm doing.
• Matter devices are already connected to the Dirigera and I can access them using that as a bridge, but I'm trying to add them directly to OpenHab.
• I have a firewall rule allowing all ICMPv6 in to the VLAN interfaces to access anywhere - I can tighten this up when its working. All blocks are logged and I cannot see any relevant packets being blocked.

When I try to add a new matter device from the IoT VLAN to the OpenHab server it all seems to go fine up to the point where OpenHab tries to talk to the device. Tracing the network comms:

• The Dirigera publishes an RA.
• The test R-Pi on the same VLAN receives this, and adds an entry to its routing table.
• OpenHab does not receive the RA.
• When OpenHab tries to contact the device from the mDNS it has no route, and fails.

So if anyone could tell me the last bit of this puzzle I'm missing I'd be really grateful - taken me quite a while to get to here but it seems so close now...

[OPNenthu]

I'm trying to understand which piece of the IPv6 puzzle is responsible for passing this RA between VLANs

Hi @barney.  RAs are not meant to pass between VLANs, normally.  That is considered a serious leak and means you no longer have L2 boundaries.  Is there a reason why you are trying to keep the hub on a separate VLAN?  It naturally wants to be on the same segment in order to form a mesh.

[barney]

Thanks for taking the time to look at this.

Is there a reason why you are trying to keep the hub on a separate VLAN?

It just seemed to be a common / recommended set up, although I admit this level of network architecture is new to me, and I may have a tendancy to over-engineer things some times...

I can see how uncontrolled boundaries could be a problem, but isn't the idea to have separate VLANs with controlled routes between them? That's what I set up in my IPv4 world:

• VLAN 10: 192.168.10.0/24 - Full Access - laptops, phones etc.
  - can access the outside world
  - can access the server on 20.16
  - can access the rest of VLAN 20/30 for testing, although I turn these firewall rules off when not being used
  
• VLAN 20: 192.168.20.0/24 -  General internal only things
  - no outbound access to the internet
  - fixed server at 192.168.20.16, running OpenHab, Lyrion Music Server etc
  - firewall rule to let 192.168.20.16 access VLAN 30 to get camera footage
  - bunch of R-Pi LMS players
  - some other stuff
  
• VLAN 30: 192.168.30.0/24 -  Cameras
  - no outbound access to the internet or any other subnet
  

So the server on 20.16 is the main thing running 24/7, and has defined firewall rules to let it access other stuff like the cameras on VLAN 30.

When I started adding smart devices it seemed to make sense to create a new VLAN for them, so I:

• created VLAN 40 and stuck the Dirigera on that
• linked up a bunch of devices using that as a hub
• added a new firewall rule to let the OpenHab server access the Dirigera

All good - I still have no IPv6 anywhere (the Dirigera has one but that's not anywhere in my set up) and OpenHab only has to talk to the Dirigera which acts as a hub for the matter devices beyond it, and the thread/matter mesh network is all in VLAN 40.

I could have left it there (and maybe I should have!), but sometimes when the Dirigera restarts it tends to lose stuff and I have to set it up again, so I wanted to use that only for onboarding the matter devices then talk to them directly from the OpenHab server, which started my long road into IPv6...

I could put OpenHab on a separate server inside VLAN 40, but I'd prefer not to run a separate server just for that if I don't need to. Also, it would mean I'd need to allow routes between VLAN 30 and 40 which are currently isolated, or just merge 30 and 40 in to one.

Is the security risk with RAs that if you allow them between subnets you are allowing a rouge device to set up any route it likes? Would a solution to be to examine the specific RA and add a fixed route from OpenHab to VLAN 40, assuming the Dirigera routing stuff is permanent and does not change?

[barney]

Ah - think I've had a light-bulb moment... and I was trying to solve the wrong problem.

Internet search for Dirigera has a lot of stuff like:

The IKEA Dirigera hub acting as a rogue Thread Border Router (OTBR) and advertising its own Unique Local Address (ULA) prefix instead of using the network-assigned IPv6 prefix is a known, significant issue when integrating it into existing Matter networks.

So the question is not: "how do I get from my network to the Dirigera thread network", but: "why is the Dirigera creating its own network and not using the ULA provided".

Which I guess is not an OPNsense issue any more, so I'll follow up on that front instead.

Again, thanks for taking the time to look - I think my IPv6 knowledge has gone from "almost nothing" to "not very much", which is a definite improvement!

Cheers,

[OPNenthu]

isn't the idea to have separate VLANs with controlled routes between them?

Certainly, but I think some concepts are getting mixed up.  For instance, you probably have DHCP in your IPv4 networks set up so that clients get their IPs only from the "server" on their own network.  Clients in VLAN 20 don't get IPs from the DHCP pool in VLAN 30.

Similarly, you don't want a router (in this case that Dirigera hub is acting as an IPv6 router) handing out IPs from its own subnet to clients in a different subnet.  That's what would happen if the RAs crossed over.  The RA would contain the prefix of the router's network (VLAN 40) and that prefix would get configured onto the clients in VLAN 20.  That would put them effectively on the same network and bypass routing/firewalling (if it even works).

To be honest I don't know how these smart home devices work (I don't own any) but I assume that they want to be on a network together and possibly the hub is providing internet access for the connected matter devices (?).  I'm not sure how that works- do they have IPs on the OPNsense network at all, or do they connect wirelessly only to the hub?

In either case, personally, I'd treat them all as "IoT" things and keep them in a VLAN together.  That way they would all talk to each other freely and do what they need to do, but they could not access my private network.  I would be able to access them, however, from a more privileged network using firewall rules.

Maybe I'm oversimplifying because I'm thinking of these devices as typical clients...

[mooh]

Maybe a quick intro to Matter over Thread will help.

In its most basic form, a Thread network works by having a so-called "Border-Router" create that network (think WiFi AP). All devices - including the border-router - added to that network use IPv6 link-local addresses to communicate. The Thread network has nothing to do with the networks on your router / firewall. IPv6 is only used because of SLAAC, not to connect it to your other networks.

Communication between devices on the Thread network and your own networks is provided by the border-router. It must be connected to one of your networks so that it can be discovered via Bonjour. Use mDNS if required. The border-router will forward Matter messages between your networks and the devices on the Thread network. Although the border-router is called a router, it actually does the forwarding on the application layer, not layer 3. That's how communication with Matter over Thread IPv6 devices works just fine, even if the border-router is connected to an IPv4 ethernet connection only.

The Dirigera Hub is a border-router. So, just put the Dirigera Hub in your IoT network and make sure the computers, phones, tablets, etc you use for home automation can communicate with the border-router.

[nero355]

Maybe a quick intro to Matter over Thread will help.

All devices - including the border-router - added to that network use IPv6 link-local addresses to communicate.

Although the border-router is called a router, it actually does the forwarding on the application layer, not layer 3.

That's how communication with Matter over Thread IPv6 devices works just fine, even if the border-router is connected to an IPv4 ethernet connection only.

Thank you for the above, because I could have sworn reading the same thing about these devices in the past!


So it's like I thought :
Dump them all in your IoT VLAN and let them have fun there!

No special IPv6 stuff needed :)

[OPNenthu]

Maybe a quick intro to Matter over Thread will help.

Good post, @mooh.  Thanks!

[barney]

Thanks everyone.

I got it to work, and the short answer is I configured a gateway / route / firewall rules in OPNsense:

Gateway:
  Name: Dirigera
  Interface: VLAN40
  Address: fdb4:66c9:5838:40:6aec:8aff:fe0d:c1fe
 
Route:   
  Network: fd2c:d79a:65f9:1::/64
  Gateway: Dirigera

fdb4:66c9:5838 is my static ULA prefix
fdb4:66c9:5838:20 is the local /64 prefix for VLAN 20
fdb4:66c9:5838:40 is the local /64 prefix for VLAN 40
fdb4:66c9:5838:40:6aec:8aff:fe0d:c1fe is the Dirigera
fd2c:d79a:65f9:1 is the thread ML-EID network - internal thread network behind the Dirigera 
 
As well as the comments here, this article on Thread IPv6 addressing helped. 

More Details
 
The starting point was:

• Dirigera is running with a bunch of devices connected to it.
• IKEA Smart Home app on my phone can connect to Dirigera and see all the devices.
• Dirigera is added to OpenHab as a matter:controller.
• Dirigera is on VLAN40 (IoT) and OpenHab on VLAN20.
• mDNS-bridge is running on OPNsense actoss VLAN20 / VLAN40.

To add a matter device directoy to OpenHab I:

• Go in to the Smart Home app and put it in pairing mode - this gives me a pairing code.
• Go in to the Dirigera controller in OpenHab and add a device - enter the pairing code.

Originally this process failed with an "unable to locate the device" message. After adding the gateway / route it works.

Tracing the network traffic on the OpenHab box I saw the following:

Code Select Expand

fe80::20d:b9ff:fe57:27c9 ff02::fb MDNS 418 Standard query response 0x0000 TXT...
fdb4:66c9:5838:20:2ecf:67ff:fe51:99e6 fd2c:d79a:65f9:1:cd4:13fb:7251:3fcc WireGuard 163 Transport Data, receiver=0x047FB532, counter=11556339582847249375, datalen=69
fdb4:66c9:5838:20::1 fdb4:66c9:5838:20:2ecf:67ff:fe51:99e6 ICMPv6 211 Destination Unreachable (No route to destination)
The MDNS response contained:

Code Select Expand

Answers
    ...
    F8470964ED4A3E53._matterc._udp.local: type SRV, class IN, cache flush, priority 0, weight 0, port 5540, target BA6ED07031FF8186.local
    BA6ED07031FF8186.local: type AAAA, class IN, cache flush, addr fd2c:d79a:65f9:1:cd4:13fb:7251:3fcc
The way I read this is that:

• Putting the device into pairing mode causes it to broadcast an MDNS response on VLAN40.
• The MDNS message says "I have a device in commissioning mode at fd2c:d79a:65f9:1:cd4:13fb:7251:3fcc".
• The MDNS bridge passes this on to VLAN20, where OpenHab picks it up.
• OpenHab tries to talk directly to that device to add it using the pairing code.
• It fails as it does not know how to get to fd2c:d79a:65f9:1.

Adding the gateway / route fixes the last step, and everything is good.

Router Advertisments

The Dirigera periodically emits an RA which updates the routing table on my test R-Pi within VLAN40:

Code Select Expand

$ ip -6 route
fd2c:d79a:65f9:1::/64 via fe80::6593:10a4:85c9:9edb dev eth0 proto ra metric 100 pref medium
fdb4:66c9:5838:40::/64 dev eth0 proto ra metric 100 pref medium
fe80::/64 dev eth0 proto kernel metric 1024 pref medium
fe80::/64 dev eth0.20 proto kernel metric 1024 pref medium
default via fe80::20d:b9ff:fe57:27c9 dev eth0 proto ra metric 100 pref medium
That is the message I was expecting to get to VLAN20 to also update the routing table there, but I can see how allowing automatic routing across VLANs could be a problem. Also, as it routes via a link-local address it probably wouldn't work even if the packet did get there.

So manually configuring the gateway / route / firewall rules makes it obvious where the cross-VLAN communications are going.

Cheers

[OPNenthu]

This might come in useful in the future.  Just so I understand: the problem this solves for you is that it isolates the Thread hub (the Dirigera) to a VLAN where you can disable internet access?

Would you be able to achieve the same with a firewall rule in an IoT VLAN?

[barney]

Yep - sort of...

It isolates the Thread Border Router (Dirigera) and all the matter devices connected through it to a VLAN where I can prevent both internet access and also access to any other VLANs (general internal, cameras etc.)

Would you be able to achieve the same with a firewall rule in an IoT VLAN?

I'm not sure. I could restrict access to the Dirigera itself as it has a known IP, but I'm not sure if I could restrict access to the thread devices behind it. The RA that the Dirigera broadcasts to its VLAN adds a direct route to all devices on the ML-EID (thread network) so I'm not sure if those packets still go via the router or not, and so could be intercepted by the firewall.

As it currently stands, all I have are three firewall rules:

• IPv4 access from OpenHab to Dirigera itself.
• IPv6 access from OpenHab to the ML-EID network (fd2c:d79a:65f9:1::/64)
• mDNS between VLAN20 and VLAN40

All other access to/from VLAN40 is blocked by default except for some common DNS / NTP etc stuff, although I'm still analysing the logs and may need some reverse access from the ML-EID network back to the OpenHab server.