G'day everyone.
I am running an OPNsense HA setup (CARP) with two nodes:
- opnsense-alpha
- opnsense-beta
Both are connected to the same upstream router, a FRITZ!Box, which receives an IPv6 prefix from the ISP.
_Upstream situation_
Both OPNsense nodes are connected to the same L2 segment (WAN side). The Fritz!Box receives a delegated prefix from the ISP:
2001:9e8:1484:3080::/58
_Observed behavior_
Each OPNsense node independently requests DHCPv6 Prefix Delegation. However, the Fritz!Box assigns different delegated prefixes to each node:
- opnsense-alpha: 2001:9e8:1484:3080::/58
- opnsense-beta: 2001:9e8:1484:30c0::/58
As a result, the tracked internal networks differ:
alpha:
LAN: 2001:9e8:1484:3081::/64
DMZ: 2001:9e8:1484:3082::/64
beta:
LAN: 2001:9e8:1484:30c1::/64
DMZ: 2001:9e8:1484:30c2::/64
_Problem_
This leads to a fundamental issue:
- The two HA nodes do not share the same IPv6 prefixes
- Internal networks differ depending on which node is active
- CARP failover results in a different IPv6 network for clients
This not just breaks the expectation of transparent failover, but also confuses v6 servers (and may be even clients?) running in a the DMZ.
_Question_
Is this expected behavior with DHCPv6-PD in an HA setup?
More specifically, is there any supported way in OPNsense to achieve consistent IPv6 prefixes across HA nodes when using DHCPv6-PD? Or is this fundamentally incompatible unless:
- a static routed prefix is used, or
- only one node performs DHCPv6-PD?
_Additional notes_
WAN connectivity and PD itself work fine on both nodes. Issue only appears when combining CARP, Track Interface and DHCPv6-PD from a consumer router (Fritz!Box). This is not about broken IPv6 connectivity, but about design limitations of DHCPv6-PD in HA scenarios.
Any guidance or recommended architecture would be appreciated, thank you!
IMHO :
- This is a silly HA setup : When the Fritz!Box goes down your whole network goes down too !!
- If you remove the Fritz!Box the issue would stay the same : Your ISP would also give you two different IPv6 Prefixes !!
At least a lot of them do AFAIK because they don't check if one Customer has requested more than one IPv6 Prefix.
This kind of HA setup is more suited for Business Customers that get a Static IPv4 Address or whole Subnet + Static IPv6 Prefix.
I run two opnsense behind my fritzbox in HA with an ndp proxy. That way both devices announce the same prefix all the time, doesn't matter which of them is online. The CARP hook prevents the ndp proxy from running on both nodes at the same time.
Please note that DHCPv6-PD might be better for most environments.
HA support was a natural consequence of the stateless design and the CARP hook from when I wrote the ndp proxy. Works pretty stable for me (in combination with a Fritzbox).
https://docs.opnsense.org/manual/ndp-proxy-go.html#high-availability
This is the os-ndp-proxy-go plugin? Do I need the os-ndproxy, too?
You only need ndp-proxy-go.
The manual explains an example setup with it.
I would recommend to read it whole as ndp proxying is a different concept than DHCPv6-PD.
The magic in a HA environment is that both OPNsense learn the same on link /64 prefix from the RAs sent by the Fritzbox, and will then proxy that to all clients behind it. That means if there is a failover, the prefix will not change, its transparent.
There is no DHCPv6 on WAN needed, only SLAAC.
Another option is SLAAC for the individual systems' uplinks but static configuration for all internal networks. But that only works if you do not have to deal with dynamic changing prefixes. I avoid these like the plague.