Confused about IPv6

[luckylinux]

Attached Network Diagram.

For a bit added clarity:
- RED: segment of the IPv6 Delegation Prefix for WAN-interfaced Zones
- BLUE: segment of the IPv6 Delegation Prefix for LAN-interfaced Zones

This looks fine to me, provided the LAN clients get the CARP address as their gateway.

One thing perhaps, usually the gateway on LAN is a link-local address, so it should be preferred to use something like fe80::1 as CARP address on the LAN side and use that as gateway for the clients.
Clients like to talk to their routers using link-local addresses only.
The GUA CARP IP on LAN should then be unnecessary. It might well be that clients use the link-local addresses of the opnsense firewalls to route and then use the wrong one as their nexthop.

You should be able to find all of that out using tcpdump.

But I thought the "Purpose" of IPv6 was to avoid NAT so end-devices could directly use IPv6 "Public" Addresses.

And use the IPv6 "Privacy Extension" (temporary Addresses) for Outbound Traffic, so that they don't get Tracked / Tagged by Websites, ISP, etc.

Otherwise why bother at all with IPv6 ? Are you "mapping" the Local-Remote Network in OPNSense using NPTv6 instead ?

[bimbar]

You misunderstand, the clients use their public address as source, but the nexthop used is always link-local. Link-local is mainly for communication between clients and routers.

Which well fits the subject .

[luckylinux]

You misunderstand, the clients use their public address as source, but the nexthop used is always link-local. Link-local is mainly for communication between clients and routers.

Which well fits the subject .

>> One thing perhaps, usually the gateway on LAN is a link-local address, so it should be preferred to use something like fe80::1 as CARP address on the LAN side and use that as gateway for the clients.

So your comment was only for the CARP adress on the LAN side ? NOT Generally. Alright 

One thing that confuses me is that if I set fe80::1 for the LAN as CARP address, what am I going to use for the other Networks/VLANs in the future ? fe81::1 , fe82::1 , etc ?

Or rather use fe80:0001::1 for LAN, fe80:0002::1 for NET2, fe80:0003::1 for NET3, etc ?

[bimbar]

You misunderstand, the clients use their public address as source, but the nexthop used is always link-local. Link-local is mainly for communication between clients and routers.

Which well fits the subject .

>> One thing perhaps, usually the gateway on LAN is a link-local address, so it should be preferred to use something like fe80::1 as CARP address on the LAN side and use that as gateway for the clients.

So your comment was only for the CARP adress on the LAN side ? NOT Generally. Alright 

One thing that confuses me is that if I set fe80::1 for the LAN as CARP address, what am I going to use for the other Networks/VLANs in the future ? fe81::1 , fe82::1 , etc ?

Or rather use fe80:0001::1 for LAN, fe80:0002::1 for NET2, fe80:0003::1 for NET3, etc ?

Best case you would use fe80::1 for all networks.

[Patrick M. Hausen]

Either fe80::1 for all. Link local addresses are local to a single link (surprise!) so using the same address on all interfaces is fine.

[luckylinux]

Either fe80::1 for all. Link local addresses are local to a single link (surprise!) so using the same address on all interfaces is fine.

Ah, cool, I didn't know that multiple Interfaces could all be identified as "fe80::1" and the System would still work ...

But where are you suggesting to put this for the CARP Interface ? On the OPNSense CARP Tutorial it says to use the CARP Address as "Source Adress" in the Router Advertisement Section: https://docs.opnsense.org/manual/how-tos/carp.html#setup-router-advertisments

So far I have it set to "Automatic" and no Route is Advertised (I think it's more reliable like this than to advertise the route for the Subnet etc).

[bimbar]

Either fe80::1 for all. Link local addresses are local to a single link (surprise!) so using the same address on all interfaces is fine.

Ah, cool, I didn't know that multiple Interfaces could all be identified as "fe80::1" and the System would still work ...

But where are you suggesting to put this for the CARP Interface ? On the OPNSense CARP Tutorial it says to use the CARP Address as "Source Adress" in the Router Advertisement Section: https://docs.opnsense.org/manual/how-tos/carp.html#setup-router-advertisments

So far I have it set to "Automatic" and no Route is Advertised (I think it's more reliable like this than to advertise the route for the Subnet etc).

Usually loopback addresses are addressed by "<IPv6 address>%<interface name>", so that it works with duplicate addresses and prefixes.
By default, if you manage addresses by SLAAC, then the source address of the router advertisement is automatically the default gateway. So you should do that as the howto specifies.

[luckylinux]

Just an Update from my side (I haven't touched OPNSense for now) ....

The Issue about LAN clients losing IPv6 Connectivity after 30 Minutes / 1 Hour / etc can be "worked around" by restarting (via CRON) the WAN/WAN6 Interface on OpenWRT Router. That's really a "hack" as it will also cause loss of IPv4 Connectivity for approx. 30 seconds over 30 Minutes (1800 seconds) so approx 2% of the Time.

This would be the Hack. BOTH WAN6 and WAN Interface need to be Restarted in order for this to Work.
**ESSENTIAL** is that WAN6 Interface **MUST** be reset **BEFORE** WAN6 Interface !

OpenWRT -> System -> Scheduled Tasks
(disabled since the better Solution has been implemented, see further down the Post, and that appears to work well ... at least for now)

Code: [Select]

# !! WAN6 Interface must be reset BEFORE WAN Interface !!
#1 * * * * /usr/local/bin/restart-wan-interfaces
#0 * * * * /usr/local/bin/restart-wan6-interfaces

WAN Interface Reset Script in /usr/local/bin/restart-wan-interfaces

Code: [Select]

#!/bin/sh

# Define Interface
interface="wan"

# Echo
echo "Bringing down Interface ${interface}"

# Stop Interfaces
ifdown ${interface}

# Wait a bit
sleep 5

# Echo
echo "Bringing up again Interface ${interface}"

# Start Interfaces
ifup ${interface}
WAN6 Interface Reset Script in /usr/local/bin/restart-wan6-interfaces

Code: [Select]

#!/bin/sh

# Define Interface
interface="wan6"

# Echo
echo "Bringing down Interface ${interface}"

# Stop Interfaces
ifdown ${interface}

# Wait a bit
sleep 5

# Echo
echo "Bringing up again Interface ${interface}"

# Start Interfaces
ifup ${interface}
Not Optimal, thus I researched a bit the OpenWRT Forums for Possible Clues ...

Primary Clue: https://forum.openwrt.org/t/loosing-ipv6-upstream-after-30-minutes/21131/21
Maybe also this: https://forum.openwrt.org/t/ipv6-works-only-with-wan-in-promiscuous-mode/490/18

So I applied the following Settings:

• OpenWRT -> Network -> Interfaces -> LAN Interface
  • Min RA interval: [3 seconds]
  • Max RA interval: [600 seconds] (same as default)
  • RA Lifetime: [1800 seconds] (same as default)
• OpenWRT -> Network -> Devices -> br-lan
  • Enable multicast querier [checked]
  • Enable multicast support: [checked]
  • Enable Promiscous Mode: [disabled] (apparently only one of multicast or promiscous mode is required, thus I disabled Promiscous Mode, since Multicast is "safer" and with less Traffic)
  • Force IGMP version: [no enforcement]
  • Force MLD version: [no enforcement]
• OpenWRT -> Network -> Devices -> wan
  • Enable multicast support: [checked]
  • Enable Promiscous Mode: [disabled] (apparently only one of multicast or promiscous mode is required, thus I disabled Promiscous Mode, since Multicast is "safer" and with less Traffic)
  • Force IGMP version: [no enforcement]
  • Force MLD version: [no enforcement]

Going strong after more than 24h with IPv6 on LAN clients working ... Knock on Wood.

It seems that old versions of OpenWRT also needed to have the "Allow-DHCPv6" Firewall Rule modified and REMOVE the SRC_IP and DEST_IP (see the explanation https://forum.openwrt.org/t/ipv6-works-only-with-wan-in-promiscuous-mode/490/17). That didn't seem necessary in the latest Stable Build though (23.05).

Overall it seems that the main Things that made this work is, in essence:

• Enable Multicast
• Use shorter time/period between RA Renewals (Min RA interval especially, since Max RA interval & Max RA interval are set at their default Values)