Confused about IPv6

[luckylinux]

I spent almost the entire day trying to get IPv6 to work.

My network setup looks like this:

INTERNET <----> OpenWRT Router <---> OPNSense Router 1 (CARP Master on LAN)
                                                    <---> OPNSense Router 2 (CARP Slave on LAN)


TLDR: Now it finally started to work (IPv6 AND IPv4), although Firefox Fallback doesn't appear to be working.

At one point everything stopped working almost by itself. I couldn't even reach the OpenWRT Router from my LAN anymore using IPv4. It seemed that the whole Routing stopped working. I also tried to Manually Add Outbound NAT but to no avail.

The only things that appeared to have solved the Problem were:
- Delete the IPv4 Gateway for WAN and let OPNSense recreate it
- UNCHECK Dynamic gateway policy in Interfaces -> LAN ("This interface does not require an intermediate system to act as a gateway")

I guess IPv6 is the perfect example of "NAT is not Firewall" kind of thinking. My private (LAN) IP address is essentially my Public IP address, since the IPv6 Delegation Prefix has been requested by OPNSense.

Is that by design or did I do some misconfiguration on my end ?

My ISP gives me a /48 Subnet, so I let OpenWRT manage/allow Delegations "up to" /48, and each OPNSense is using "just" a /50. So theoretically, this should let me have 3 Routers on that switch with /50 each (because both OpenWRT and each of the OPNSense Routers is already consuming 1 IP).

There are however plenty of things that are "weird"/not optimal:

- The DHCP6 Server Subnet do NOT overlap because the WAN IP "Block" (the /50 Delegation Prefix) are 2 consecutive Blocks. So basically each Client in the LAN is getting 2 IPv6 Addresses (I'm getting like 3-4 actually + the link-local fe80:: ... I guess because the DHCP Servers reply at different speeds, so sometimes the Slave is faster). So I cannot configure CARP like this, since the Subnets don't overlap  .

- Is it normal that my Private IP is the same as my Public IP now ? It feels like IP are not tied to an Interface/Network anymore (I guess I miss NAT )

- Even though I configured (maybe badly ) Router Advertisement in "Assisted" Mode, all the Leases are listed in Services -> ISC DHCPv6 -> Leases. Is that normal ?

Thank you for your help  .

[Saarbremer]

Hi,

as a reminder:
NAT is not a feature. It's a neccessary workaround in IPv4. In IPv6 you're online with a dedicated address. Changing constantly when privacy ext. is enabled.

NAT does not provide security. Your firewall does.

DHCPv6 is not required unless you do prefix delegation. Using the flag assisted you specify that clients can basically chose between DHCP and SLAAC. Android choses SLAAC, windows choses DHCP. Hence your IP address will be listed on the leases tab.

A host with IPv6 connectivity usually has a link-local adress and 1-n global unique adresses. Privacy extensions do generate one every hour and keep the old one. With assisted you may have a SLAAC and a DHCP assigned adress. In a standard /64 network this can be neglected.

A transfer net can be set up without GUAs. When all hosts are routers link-local adresses are fine. However, to access internet (e.g. for updates) they would require one.

Hope that helps to come up with better expectations

[luckylinux]

So basically, it's working as it should ?

I just checked ... Actually my Android Phone stopped having Internet Access after this IPv6 Configuration. Weird ... At the very least it should fallback to IPv4 I would expect.

I just checked 2 GNU/Linux Hosts and they have no Problem.

But maybe it's because the WiFi System (UNIFI) doesn't have IPv6 configured on each Access Point and the Controller ?

[bartjsmit]

Modern browsers use happy eyeballs to decide between v4 and v6 https://datatracker.ietf.org/doc/html/rfc8305

Unifi AP's don't need to support IPv6 management to carry IPv6 traffic. Recent AP's support multicast well enough for it to work.

Bart...

[luckylinux]

I just checked again on the Android Phone ... IPv4 seems to work again. IPv6 doesn't work at all.

But my goodness the UNIFI Controller ... getting IPv4 to work is as simple as just saying "Just query the [main] DHCP Server".

For IPv6 on the other hand 

Not sure how most people do it ... Do you make 1 VLAN for each "Service Group", and on that VLAN you assign a separate say /54 delegated DHCPv6 Prefix, similarly to what you would do in IPv4 by assigning e.g. 10.10.0.0/20 or 172.20.0.0/20 or 192.168.0.0/24 or similar ?

So basically "every service is part of the [ISP delegated] IPv6 prefix /48 INTERNET-wise, but then the individual subnets are identified locally by restricting/shrinking the subnet even more, say /56 LOCALLY" ?

[Patrick M. Hausen]

So basically "every service is part of the [ISP delegated] IPv6 prefix /48 INTERNET-wise, but then the individual subnets are identified locally by restricting/shrinking the subnet even more, say /56 LOCALLY" ?

I don't parse that question 100%, but for one general statement: in IPv6 every broadcast domain (i.e. "Ethernet like network) is a /64. Always.

Here in Germany customers of German Telekom get a /56, which means you can have 256 separate VLANs with one /64 each.

If you are in the lucky position to get a /48 from your ISP that means you can simply (not walk into Mordor) e.g. have 256 locations in your corporate networks with then 256 VLANs each.

You never assign a longer prefix than /64 to a single interface.

HTH,
Patrick

[muchacha_grande]

Here in Argentina we are not so lucky. We get a /64 network from our ISP and if we want VLANs we need to subnet it.
I manage to assign /80 to each of my VLANs and forget to have IPv6 for smart phones.

[Saarbremer]

But my goodness the UNIFI Controller ... getting IPv4 to work is as simple as just saying "Just query the [main] DHCP Server".

For IPv6 on the other hand 

This is not a unifi forum. Why don't you just buy a USG?

BTW: You assign /64 subnets for segments. You can however delegate more prefixes to more subnets reachable within that segment via routers.

[luckylinux]

So basically "every service is part of the [ISP delegated] IPv6 prefix /48 INTERNET-wise, but then the individual subnets are identified locally by restricting/shrinking the subnet even more, say /56 LOCALLY" ?

I don't parse that question 100%, but for one general statement: in IPv6 every broadcast domain (i.e. "Ethernet like network) is a /64. Always.

Here in Germany customers of German Telekom get a /56, which means you can have 256 separate VLANs with one /64 each.

If you are in the lucky position to get a /48 from your ISP that means you can simply (not walk into Mordor) e.g. have 256 locations in your corporate networks with then 256 VLANs each.

You never assign a longer prefix than /64 to a single interface.

HTH,
Patrick

I read things about /64 Subnet Size with IPv6 and must say I'm quite Confused there as well.

I'm pretty sure I read /64 is the MINIMUM Subnet Size [for Delegation ?].

But then it's true that when my OPNSense Firewalls get their "own" WAN IP from OPNSense, then they are getting a /128 (single Host).

I also read some of your Posts on this Forum Patrick (not sure if directly related to my current Issue) about CARP and IPv6.

I am getting an IPv6 /48 Prefix

2aXX:XXXX:XXXX::/48 -> Delegation from my ISP

2aXX:XXXX:XXXX::1/48 -> What OpenWRT Router can manage/is managing on the LAN Side (Public IP)
fdXX:XXXX:XXXX::1/48 -> What OpenWRT Router can manage/is managing on the LAN Side (Local ULA)

2aXX:XXXX:XXXX:4000::/50 -> goes to OPNSenseRouter1 (Public IP)
fdXX:XXXX:XXXX:4000::/50 -> goes to OPNSenseRouter1 (Local ULA)

2aXX:XXXX:XXXX:8000::/50 -> goes to OPNSenseRouter2
fdXX:XXXX:XXXX:8000::/50 -> goes to OPNSenseRouter1 (Local ULA)

Here the first Problem / Non-Optimum: due to CARP, I'm "losing" half the available IP Subnets AND Duplicating all IPs (due to DHCP Server timing, each host might get 1 IP from OPNSenseRouter1 and 1 IP from OPNSenseRouter2).

Then I can split each /50 into (2^(64-50-1) = 8192 ?) Separate /64 Networks, which I might as well group into VLANs (sigh ... due to Mellanox ConnectX2/3 Limitations, I believe a Maximum of Only 125-128 VLANs is supported anyway).

And of course each of these ranges, I will most likely split into 2, similarly to what I do with IPv4 (lower part for statically assigned / reservation of IP Addresses, higher part for dynamically assigned IP Addresses).

But isn't there a way for the 2 OPNSense Routers to work together "on the same Dedication Prefix" say /49 (instead of 2 x /48 with all the duplication that comes with it) ?

I could only get OpenWRT to give out the requested Delegation Prefix in "Basic Mode".
In "Advanced Mode", either specifying the "ia-pd 1" or "ia-na 1" etc, does not work at all.

And I guess it's a feature, not a bug, that the 2 OPNSense instances cannot get the same Delegation Prefix from the upstream Router, or ?

[Patrick M. Hausen]

If you really get a /48 from your ISP isn't that a static one? So configure all interfaces statically. CARP works in this case. I don't know if prefix delegation and a HA cluster play together well, but as a guess I doubt it.

[luckylinux]

If you really get a /48 from your ISP isn't that a static one? So configure all interfaces statically. CARP works in this case. I don't know if prefix delegation and a HA cluster play together well, but as a guess I doubt it.

To be honest I don't know if IPv6 Prefix is Static. I purchased a separate static IPv4 from my ISP, but I don't know about the "default" IPv6 Subnet.

I'd also guess that HA with Prefix Delegation could cause issue.

So on both OPNSense Routers, you propose for the WAN that I should select a "Static IPv6" and use the /49 Subnet on both, with IP maybe ::11 for Router1 and ::12 for Router2 ?

Then for LAN (and all other interfaces: DMZ, WiFi, VLAN_XXX), I setup as "Track Interface" as usual with id 0,1,2,3,.... ?

[Patrick M. Hausen]

So on both OPNSense Routers, you propose for the WAN that I should select a "Static IPv6" and use the /49 Subnet on both, with IP maybe ::11 for Router1 and ::12 for Router2 ?

Then for LAN (and all other interfaces: DMZ, WiFi, VLAN_XXX), I setup as "Track Interface" as usual with id 0,1,2,3,.... ?

No, no no ... for WAN there is obviously some transfer network in place so you ask your ISP what exactly to configure.

And you never configure anything but /64 (or for WAN in some cases /128) on a single interface.

Then you pick one /64 from your /48 for LAN and configure one address from that prefix on the master and another address from the same prefix on your slave. This is just like IPv4 - both nodes share a single network on each interface.

[luckylinux]

So on both OPNSense Routers, you propose for the WAN that I should select a "Static IPv6" and use the /49 Subnet on both, with IP maybe ::11 for Router1 and ::12 for Router2 ?

Then for LAN (and all other interfaces: DMZ, WiFi, VLAN_XXX), I setup as "Track Interface" as usual with id 0,1,2,3,.... ?

No, no no ... for WAN there is obviously some transfer network in place so you ask your ISP what exactly to configure.

And you never configure anything but /64 (or for WAN in some cases /128) on a single interface.

Then you pick one /64 from your /48 for LAN and configure one address from that prefix on the master and another address from the same prefix on your slave. This is just like IPv4 - both nodes share a single network on each interface.

I'm giving them a call right now.

I understand your second part of the reply. Not so much the first one, sorry  .

>> No, no no ... for WAN there is obviously some transfer network in place so you ask your ISP what exactly to configure.

Am I NOT supposed to use a part of the Prefixed Subnet they assign me ? Because this is what I'm currently doing (and mentioned in the first post that my Private IP = Public IP, since no NAT).

EDIT 1: on the phone, the "IT Expert Consultant" of the ISP told me that he does NOT know how IPv6 works. He asked a colleague who told him that he never saw IPv6 Addresses change, but he cannot promise that it will never change. Nice to see when ISP IT Experts know even less about IPv6 than me as a Homelabber . "IPv6 is too complicated to setup I will never do it" he told me .

This was Technical Support, not even Customer Support  .

[Patrick M. Hausen]

Frequently ISPs use a single IPv6 address for the external connection of their customers in addition to a delegated prefix.

For example I currently got 2003:a:d7f:d938:f690:eaff:fe00:ca67/64 on WAN while my delegated prefix is 2003:a:d59:3800::/56.

Like in IPv4 this is commonly called a transfer network, because it is not used for any services but simply so that the routers have dedicated addresses. Different from IPv4 it is not strictly necessary. Routing works fine over just link-local addresses.

Whatever is the case your ISP should be able to tell you but it seems they are not. 

From that /48 of yours you then pick an arbitrary /64 for LAN and assign e.g. ::1 to your first node and ::2 to the second. For CARP you should pick a link-local address that will then be announced to all your clients as the default gateway via router advertisments. E.g. fe80::1

Essentially IPv6 is way simpler than IPv4. Only forget you must what you have learned 

Try prefix delegation ("track interface) on both nodes and set the prefix hint to "0" on both. They should have a single common /64 on LAN - only then can you proceed with CARP etc.

[luckylinux]

Frequently ISPs use a single IPv6 address for the external connection of their customers in addition to a delegated prefix.

For example I currently got 2003:a:d7f:d938:f690:eaff:fe00:ca67/64 on WAN while my delegated prefix is 2003:a:d59:3800::/56.

Like in IPv4 this is commonly called a transfer network, because it is not used for any services but simply so that the routers have dedicated addresses. Different from IPv4 it is not strictly necessary. Routing works fine over just link-local addresses.

Whatever is the case your ISP should be able to tell you but it seems they are not. 

From that /48 of yours you then pick an arbitrary /64 for LAN and assign e.g. ::1 to your first node and ::2 to the second. For CARP you should pick a link-local address that will then be announced to all your clients as the default gateway via router advertisments. E.g. fe80::1

Essentially IPv6 is way simpler than IPv4. Only forget you must what you have learned 

Try prefix delegation ("track interface) on both nodes and set the prefix hint to "0" on both. They should have a single common /64 on LAN - only then can you proceed with CARP etc.

I am getting a /128 and a /64 for use on the OpenWRT Router itself plus the /48 Prefix for LAN Delegation:
LAN Interface: 2aXX:XXX4:XXXX::1/48
WAN Interface: 2aXX:XXX0:XXXX:....:12c8/128
WAN Interface: 2aXX:XXX0:XXXX:....:2a4/64

Note that the Prefix for Delegation is different than the Public IP of the OpenWRT Router on the WAN Interface.
One is 2aXX:XXX4:... (for Prefix Delegation), the other one is 2aXX:XXX0: ... (for use by the OpenWRT WAN Interface itself).

Downstream the OPNSenseRouter1 gets ::7 and OPNSenseRouter2 gets ::8.

I might need to change those numbers since now ::1 is already taken by OpenWRT (I could change that though) ...

The "Reason" (at the time) was that I once had OPNSenseRouter1 IPv4 on 192.168.1.1 and 192.168.1.3 - 192.168.1.4 - 192.168.1.5 (and maybe 192.168.1.6) were already taken for the DNS Servers.

So when I introduced CARP with a second OPNSense (OPNSenseRouter2) I needed 3 addresses and therefore I picked 192.168.1.1 for the Virtual/CARP/Common IP (since that's the Gateway it's statically configured on many Client Devices and it's better not to change that) and 192.168.1.7/192.168.1.8 for the Individual Routers.

The only Technical Information they provided is this (translated by Google Translate):

Code: [Select]

Thank you for the conversation

here with what we have on Ipv6, hope it can help a little

IPv6

     Certain DHCPv6 settings must be set up on your router.
         SLAAC (point to point link between your router and our GW router).
         DHCPv6 with prefix delegation (DHCPv6-PD) (used to retrieve a delegated /48 prefix that can be distributed on the lan side of your router).
         Stateful Address Assignment | IA-NA = 1 (on)
         Identity Association for Prefix Delegation | IA-PD = 1 (on)
         Non-temporary address | IAID for NA = 1 (on) instead of 0 (off) which, according to our experience, will be the default setting in most DHCP client setups.
         We recommend using the DUID-LL [DUID type 3] algorithm (Algorithm used on routers issued by us).
         We recommend setting a static DUID-LL value (if possible) to avoid the value changing between restarts of your CPE (router) device.
     We assign you a /48 via DHCPv6-PD (Prefix Delegation).
         The /48 route is inserted into our routing table by our DHCPv6 relay.
         Without DHCPv6-PD you will not be able to use your /48 prefix.
     Our GW routers are set up to send router advertisements every 360 seconds (6m).

 
DUID (DHCP Unique Identifier)

 

     DUID-LLT: The Link-Layer address of one of the device's network interfaces, concatenated with a timestamp [RFC 2131].
     DUID-EN: An Enterprise Number plus additional information specific to the enterprise [RFC 2131].
     DUID-LL: The Link-Layer address of one of the device's network interfaces [RFC 2131].
     DUID-UUID: Used in situations where there is a Universally Unique IDentifier (UUID) stored in a device's firmware settings [RFC 6355].
     DUID-V6ADDR: "This document defines a new DHCPv6 Unique Identifier (DUID) type that contains a single 128 bit IPv6 address. Makes it possible for devices to use suitably-derived unique IPv6 addresses to identify themselves to DHCPv6 servers"


So I setup, on the OpenWRT Interface Properties, "Client ID to send when requesting DHCP" according to DUID-LL as they Reccomend. Nothing changed when I hit refresh though (neigher IPv4 nor IPv6, I guess it makes sense since it's the same as the default).

Unfortunately OpenWRT is quite confusing with wan/wan6 Interface, whereby wan (supposedly IPv4) contains some IPv6 Settings and wan6 (supposedly IPv6) contains some IPv4 Settings.

OPNSense itself has quite some quircks ... When asking upstream for /50 Delegation, there is no way to see that it actually gets it. In Interfaces -> Overview I always get a /128 plus a /64 (no matter what I do) plus a weird fe80::XXXXXX/64 which I did NOT configure anywhere for the WAN Interface (is that the link-local maybe ? Upstream Gateway is fd34:XXXX:XXXX::1 (OpenWRT Local ULA).

The only way I could see if OPNSense actually gets the /50 prefix is by looking at the OpenWRT DHCPv6 Leases page.