CALL FOR TESTING: Multi-dhcp6c for multi-WAN IPv6

[franco]

Hello,

This builds on 26.1.3 and the previous topic https://forum.opnsense.org/index.php?topic=50401.0 and is mainly intended for multi-WAN users with multiple DHCPv6 WAN configurations, because there are some downsides to using only one daemon for all WANs.

This requires the 26.1.3 version to apply cleanly.

# opnsense-patch https://github.com/opnsense/core/commit/c5cb86b

(and reboot)

Why do this? When you have two WANs that both get IPv6 information via DHCPv6 they are currently merged into a single configuration. If one of the two interface is being reconfigured the only dhcp6c process gets a SIGHUP to restart wich restarts IPv6 connectivity for both WANs instead of only the right one. It will at least remove side effects from multi-WAN environments on this level.

The other addition is complete control over prefix association as described in the linked ticket: https://github.com/opnsense/core/issues/7647

This may at least make a few AT&T users happy.  :)

Things that would be useful to know:

1. Is this a fire and forget change first and foremost or does it introduce compatibility issues?
2. Is the new PD association feature working as intended?
3. Are you still using config file overrides and why?

The plan beyond this is likely to remove the "advanced" configuration mode for DHCPv6. I'm not overly fond of the "override" mode but it seems more useful than advanced so it's likely going to stay. So we can merge more advanced settings into the "basic" configuration. In a mid-term MVC/API world it's easier to hide more advanced settings, too.


Cheers,
Franco

[jrichey98]

Franco,

I recently installed a fresh install of opnsense 26.1 on a router. I have been running dual-wan for years (ATT & Spectrum), but with ISC DHCP for v6/v4, and figured it was time for a fresh config.

The default was DNSMasq, I couldn't get router advertisements to work or see leases (though ipv4 was working, ipv6 was not), so I switched over to KEA / RA. DHCPv4/6 are working well and assigning leases and RA daemon is configured as Managed (A+O) and working great. I get a warning that I should be using a /64 it doesn't seem to effect anything.

I also switched over to the new firewall rules.

My networks are as follows:
WAN1: DHCP / DHCPv6
WAN2: DHCP / DHCPv6
LAN: Static - 172.20.0.1/24 / fd00::1/112
*probably should make it a /120 to match number of addresses, or a /96 so I can 1:1 the values in the last octet to the last group.

** Note: Image tags not working for me, included links **

My Home screen: <link>


NAT is configured manually, and I have the following rules: <link>


I have the following gateways: <link-int> <link-group>



If I wait about couple minutes a link-local IP (fe80 will show up) on WAN1, and I can then manually start the gateway monitor for WAN1_DHCP6: <link>


Firewall: <link1> <link2>



SYSLOG : <link>
Config : <link>
*Password hash removed

I tried the patch and didn't notice any different behavior before.

Post-Patch SYSLOG: <link>

I do have a few opinions on Multi-WAN configs:
- I like using a private range for IPv6 and NAT'ing, because it means IPv4 works exactly the same as IPv6 which makes it simple to manage.
- When traffic is being redirected to different gateways, tracking an interface seems problematic.
- Defaulting to a net in the private IP space (fc00/7), and doing a One-to-One NAT, is probably the best solution when using multiple WAN/Gateways (I personally just NAT to the interface address, but you have the IPs with v6 so might as well use them).

These are just my opinions, but IMHO IPv6 keeps pretending they engineered all the use-cases away for translation, but I just think they cause more problems trying to throw away the toolbox.

In any case, I'd love to get my router's second ISP (Spectrum) up and working, and both of them without manually intervention (hitting start on the gateway monitor).

This is my home router and not a production system, and i haven't added my lab nets yet so it's pretty barebones. If you want me to test anything let me know.

[Maurice]

I applied the patch to 26.1.3. Unfortunately, it doesn't work for me. dhcp6c fails to acquire an address (IA_NA) on the secondary WAN, and sometimes fails completely (no address on both WANs and no prefix delegation on WAN 1).

One thing I noticed in packet captures is that the IAID is now set to 0 for both WAN interfaces. That's not supposed to happen, each interface must have a distinct IAID. Not sure whether this is the root cause, but it's plausible because in my case, both WAN interfaces are served by the same upstream DHCPv6 server.

Without the patch, dhcp6c uses a distinct IAID for each interface. Had to roll back the patch. Happy to test again when unique IAIDs are back.

Config WAN 1:

Code Select Expand

<if>vtnet0</if>
<descr>WAN_GPON</descr>
<enable>1</enable>
<lock>1</lock>
<blockpriv>1</blockpriv>
<blockbogons>1</blockbogons>
<mtu>1492</mtu>
<ipaddrv6>dhcp6</ipaddrv6>
<dhcp6-ia-pd-len>8</dhcp6-ia-pd-len>

Config WAN 2:

Code Select Expand

<if>vtnet1</if>
<descr>WAN_LTE</descr>
<enable>1</enable>
<lock>1</lock>
<blockpriv>1</blockpriv>
<blockbogons>1</blockbogons>
<ipaddrv6>dhcp6</ipaddrv6>
<dhcp6-ia-pd-len>none</dhcp6-ia-pd-len>

Cheers
Maurice

[franco]

@jrichey98

> The default was DNSMasq, I couldn't get router advertisements to work or see leases (though ipv4 was working, ipv6 was not), so I switched over to KEA / RA. DHCPv4/6 are working well and assigning leases and RA daemon is configured as Managed (A+O) and working great. I get a warning that I should be using a /64 it doesn't seem to effect anything.

I haven't heard this before but good to know.  Don't know what is wrong though.  Need to keep this under observation.

I'm also unsure why your WAN DHCPv6 seems to misbehave in the standard case.  This patch is only designed to allow to manage associations per interface in a fine-grained fashion.


@Maurice

Thanks!  Exactly why we're here testing.

I have to say this is somewhat expected against the same DHCPv6 server at the price of yielding full control of the associations to the user. It's difficult to support both at the same time. The indexing code is a bit whacky in general:

https://github.com/opnsense/core/blob/10c4d20dbc009ca73e201c80e4bb2f043b9416f4/src/etc/inc/interfaces.inc#L2920-L2940

IMO this isn't rooted in any type of reality -- it just tries to unbreak what you describe in a crude way and there is no (elegant) way to prevent overlap in manual settings if we keep doing this.

NA has that same issue now I guess.  Also fixable with an additional setting.

Would it help if we split the DUID like VyOS does? :>

https://github.com/vyos/vyos-build/blob/current/scripts/package-build/wide-dhcpv6/patches/wide-dhcpv6/0023-dhcpc6-support-per-interface-client-DUIDs.patch

Because that was on my wishlist...


Cheers,
Franco

[Maurice]

Why not just keep the existing logic of sequentially numbering the IAIDs as the default / automatic behavior (for IA_NA and IA_PD)?
In case of manual configuration, preventing duplicate IAIDs is under the control of the user. What we definitely mustn't have are duplicate IAIDs as the default.

An additional setting for manually specifying the IAID for IA_NA might be nice to have. Or we could just use the same for IA_PD and IA_NA. I currently can't think of a use case where these would have to be distinct, but I'm sure someone will come up with one. :)

And please use the keyword "IAID" in the GUI. It took me embarrassingly long to figure out that this is what's meant by the new setting.

Multiple DUIDs as a workaround for duplicate IAIDs seems overkill to me. But if there is actual demand for multiple DUIDs for special use cases, sure, why not.

Cheers
Maurice

[franco]

> Why not just keep the existing logic of sequentially numbering the IAIDs as the default / automatic behavior (for IA_NA and IA_PD)?

Because they are purely random and wrong.  I don't mind another look or proposal.

> In case of manual configuration, preventing duplicate IAIDs is under the control of the user. What we definitely mustn't have are duplicate IAIDs as the default.

But that only matters for IAID+DUID pairs when the multi-WAN connections are going to the exact same server.

I must admit the IAID changes are not for multi-WAN and we can test without them to go forward in 26.1.x regardless. Let me prepare a patch for that.

> An additional setting for manually specifying the IAID for IA_NA might be nice to have. Or we could just use the same for IA_PD and IA_NA. I currently can't think of a use case where these would have to be distinct, but I'm sure someone will come up with one. :)

IA_NA is so rudimentary implemented that I don't see much need to overcomplicate it now. It is basically like you suggest but with the random/illogical derivation of the default IAID.

> And please use the keyword "IAID" in the GUI. It took me embarrassingly long to figure out that this is what's meant by the new setting.

Sure, I'll try to reword.  The GUI isn't really helping in this regard historically.

> Multiple DUIDs as a workaround for duplicate IAIDs seems overkill to me. But if there is actual demand for multiple DUIDs for special use cases, sure, why not.

I think I disagree here.  There isn't a strong demand, but with multi-dhcp6 it's a possibility to leverage.


Cheers,
Franco

[demyers]

- I like using a private range for IPv6 and NAT'ing, because it means IPv4 works exactly the same as IPv6 which makes it simple to manage.

At the risk of going off topic, I have to say that after many years of running multi-WAN (most with pfSense but more recently with OPNsense) I completely agree with you. Using NAT to keep your LAN prefix from changing makes failover smoother, especially with ISPs that change your prefixes whenever they feel like it.

Note, however, that if you use a private range, your LAN clients will tend to prefer IPv4 connections (likely due to the use of getaddrinfo(3) and its compliance with RFC 3484). What I've ended up doing is making up my own public IPv6 prefix for use on my LAN so my clients prefer IPv6.

We're just breakin' all the (IPv6) rules!

[Maurice]

Because they are purely random and wrong.

Random yes, but wrong? I'm not aware of any RFCs requiring IAIDs to have semantic meaning or structure. Any 32-bit unsigned integer is fine.

I don't mind another look or proposal.

Just keep in mind that changing IAIDs can break existing setups. If DHCPv6 static bindings are configured on the upstream DHCPv6 server, an IAID change can cause a mismatch. I recently had that issue in the real world when OpenWrt changed their IAID calculation logic.

But that only matters for IAID+DUID pairs when the multi-WAN connections are going to the exact same server.

Which is not that uncommon, e. g. if you have two lines from the same ISP. Also, duplicate IAIDs (in the context of a given DUID and IA type) are a clear violation of the DHCPv6 specifications (RFC 9915), which I think we shouldn't do for no good reason.

I must admit the IAID changes are not for multi-WAN and we can test without them to go forward in 26.1.x regardless.

I agree, it's a good idea to split these two issues.

Let me prepare a patch for that.

I'll be happy to test it.

Sure, I'll try to reword.

Thanks a lot!

There isn't a strong demand, but with multi-dhcp6 it's a possibility to leverage.

I have no strong opinion about multiple DUIDs. While it doesn't seem intuitive (a DUID literally Uniquely IDentifies a Device, not a DHCPv6 client instance), I see no harm in it. And there might be use cases where it's actually helpful.

Cheers
Maurice


@demyers

At the risk of going off topic

Please don't, this is a CFT.

[franco]

Here's the isolated change (will apply to a clean 26.1.3/4)

# opnsense-patch https://github.com/opnsense/core/commit/2db56bfee

> Random yes, but wrong? I'm not aware of any RFCs requiring IAIDs to have semantic meaning or structure. Any 32-bit unsigned integer is fine.

Permanently mangled then. Deleting an interface can negatively affect the default ID generation.

> Just keep in mind that changing IAIDs can break existing setups. If DHCPv6 static bindings are configured on the upstream DHCPv6 server, an IAID change can cause a mismatch. I recently had that issue in the real world when OpenWrt changed their IAID calculation logic.

I don't doubt that.  In practice the default WAN is always 0 though.

> Which is not that uncommon, e. g. if you have two lines from the same ISP. Also, duplicate IAIDs (in the context of a given DUID and IA type) are a clear violation of the DHCPv6 specifications (RFC 9915), which I think we shouldn't do for no good reason.

We do not / cannot know that so we always mitigate it seems.  The reason to change it is a good one, but the repercussions are definitely there too.

>  have no strong opinion about multiple DUIDs. While it doesn't seem intuitive (a DUID literally Uniquely IDentifies a Device, not a DHCPv6 client instance), I see no harm in it. And there might be use cases where it's actually helpful.

It's not often asked for but the use case is there:

https://forum.opnsense.org/index.php?topic=23012.0
https://forum.opnsense.org/index.php?topic=46607.0

Embedding the DUID into the configuration also has some benefits for maintenance regardless of multi-WAN.


Cheers,
Franco