IPv6 default route lost / ICMP6 RA not received

[loredo]

Hello,

I am facing an issue to keep my IPv6 connectivity up and running as the default route to fe80::1 will disappear.

When sniffing the traffic using "tcpdump -pni igb0 icmp6", it seems the ICMP6 RA package is not received on the WAN interface. However, sniffing in promiscuous mode by "tcpdump -ni igb0 icmp6" will show the incoming RA and also NDP will work just fine then. In that case the default route will be put back into the routing table.

Similar issue was already discovered on pfSense a couple of month ago:
https://redmine.pfsense.org/issues/8611

Is there any other fix besides a workaround to permanently put the interface into promiscuous mode by "ifconfig igb0 promisc" during bootup?


Thanks,
Julian

[loredo]

What I can also say is that it has something to do with the gateway monitoring part as well.
Editing the auto-generated gateways to enable monitoring seems to promote the missing RA announcement effect. It wouldn't help then to just disable gateway monitoring again, instead one needs to completely delete the respective gateway to have it auto-generated again. afterwards, the IPv6 connectivity will be stable even without promiscuous mode enabled.

Any idea how I could still enable gateway monitoring w/o affecting my default IPV6 route? I noticed that an explicit route to the gateway IP via the interface itself will be put into the routing table, could it be that this has a negative affect?

[loredo]

Not sure if anybody is actually reading my posts  but I will simply continue to report about the progress...

I believe I was able to come a little closer to resolve this issue by disabling "Block private networks" on the WAN interface. However, the description does not say anything at all about what it actually does on the IPv6 layer:

When set, this option blocks traffic from IP addresses that are reserved for private networks as per RFC 1918 (10/8, 172.16/12, 192.168/16) as well as loopback addresses (127/8). This option should only be set for WAN type interfaces that use public IP address space.

Could somebody describe what will be done for the IPv6 layer when enabling that option?
The connection will stay up and running after having this disabled, also after rebooting the device. Editing the gateways to enable monitoring is also not a problem after all.

On the other hand, "Block bogon networks" is working just fine for IPv6. I could also add a custom firewall rule for IPv4 on the WAN to block traffic for RFC1918 + 127/8 without any issues.

To me it seems that some kind of src/dst IPv6 addresses are blocked for ICMP6 traffic that is required for correct router announcement+solicitation as well as neighbourhood discovery when using fe80::/10 as the default route.

There is also ff02::1:ff00:0/104 multicast in the game when the ISP router is sending RA.
Not sure why enabling promiscuous mode would also help here though...

I hope this analysis result will help to improve on the IPv6 support. This is the relevant tcpdump log for anyones' reference:

Code: [Select]

10:40:38.571045 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:40:38.577288 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:40:44.950916 IP6 fe80::1 > ff02::1: ICMP6, router advertisement, length 32
10:41:23.601725 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:41:23.607242 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:42:09.630361 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:42:09.637501 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:42:26.219263 IP6 2a02:810d:0:1e::1 > 2a02:810d:0:1e:1ca8:XXXX:XXXX:XXXX: ICMP6, neighbor solicitation, who has 2a02:810d:0:1e:1ca8:XXXX:XXXX:XXXX, length 32
10:42:54.683225 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:42:54.691631 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:43:03.900589 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor solicitation, who has fe80::20d:b9ff:XXXX:XXXX, length 32
10:43:03.900709 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor advertisement, tgt is fe80::20d:b9ff:XXXX:XXXX, length 24
10:43:39.530103 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:43:39.535265 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:44:24.538158 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:44:24.548990 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:45:09.603706 IP6 fe80::20d:b9ff:XXXX:XXXX > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:45:09.610927 IP6 fe80::1 > fe80::20d:b9ff:XXXX:XXXX: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
10:45:23.847963 IP6 2a02:810d:0:1e::1 > 2a02:810d:0:1e:1ca8:XXXX:XXXX:XXXX: ICMP6, neighbor solicitation, who has 2a02:810d:0:1e:1ca8:XXXX:XXXX:XXXX, length 32

What's also interesting to see here is that the neighbor solicitation for my WAN public IP address is still not answered by the OPNsense box (but will still be accessible...). Might be that the ISP router doesn't care as long as the link-local neighbor solicitation is still successful.

[marjohn56]

I'll try and look at this over the coming week end and try and replicate it.

[loredo]

Thank you so much!

Let me know if you want me to do any further tests here, happy to provide any additional info now that I know the procedure to have this working for the moment.

[marjohn56]

So far I've not been able to replicate this, but then again I am running 19.1b.


Would you be in a position to try 19.1b?

[GDixon]

I'm using the beta you mentioned and have the same problem. I may be a spectrum/comcast problem possibly.

[Kingrat]

Im on Spectrum as well in a TWC legacy area. IPV6 has always worked fine for me after a fresh reboot but I had problems with it not working after basically any hiccup that caused the lan interface to flap eg. disconnecting or updating my switch. I had to check "Prevent Release" under my WAN interface and that fixed that issue for me. I'm even using a /56 prefix and have no more problems with IPV6 currently.

[GDixon]

my mistake, spectrum bought out bhn here in the Tampa area.

IPv6 is very problematic.

What are your other settings for spectrum?

 I'm also using /56. I tried a /64 but If i use /64 then NTP flips out and Dhcp6 no longer runs and anything else (for example /60) makes radvd puke about needing to see a /64

a /56 seems to be a starting place for everything to run and not puke. But still no IPv6

It would be nice to have IPv6 working but theres some kind of problem between Spectrum and OPNsense, they no like one another it seems.

 I give up maybe another day.

[loredo]

Would you be in a position to try 19.1b?

I fear I can't offer that, need working connectivity for work.

I'm using the beta you mentioned and have the same problem. I may be a spectrum/comcast problem possibly.

I don't think so. My cable provider is Vodafone Kabel Deutschland and their own CPE is working fine.

Having "Block private networks" disabled on the WAN interface, IPv6 is running rock solid now here in every sense. Enabling it will break it again. I wasn't able to find the root cause as nobody would answer my questions from below, but eventually I have found a workaround that works.

[loredo]

Oh and what I also learned:

If you play a lot with the settings, the DHCP6 server of your provider will most certainly not answer back the way you would normally expect. The server will remember IP ranges assigned to you already and will only respond them again if the client DHCP UID will match and the requested IP range size will be the same.

The safest way is to have IPv6 disabled for something like 2 hours and then re-enable it with the settings that should work:

1. Disable IPv6 on the WAN interface and any LAN interfaces.
2. Disable "Block private networks" on WAN.
3. Optional: Manually implement FW rules to still block RFC1918 IPv4 addresses on the WAN interface.
4. Set a static DHCP UID in Interfaces > Settings > "DHCP Unique Identifier"
5. Wait a couple of hours for potential old DHCP6 leases to time out.
6. Optional: If you use DHCP for IPv4 connectivity on the WAN interface, it is good to set "Reject Leases From" to avoid receiving the wrong IPv4 lease during reboot of the cable modem. According to the DOCSIS standard it should always be IP 192.168.100.1 but it might be different for your specific modem (or router in bridge mode).
7. Enable IPv6 on WAN the following way:
- Request only an IPv6 prefix: DISABLED
- Prefix delegation size: 56 (might be different, ask your provider)
- Send IPv6 prefix hint: ENABLED
- Directly send SOLICIT: ENABLED
- Prevent release: DISABLED
- Enable debug: DISABLED
- Use IPv4 connectivity: DISABLED
- Use VLAN priority: DISABLED
8. Click on "save" _and_ also apply changes.
9. Enable IPv6 on LAN the following way:
- IPv6 Configuration Type: Track Interface
- IPv6 Interface: WAN
- IPv6 Prefix ID: 0 (if that's the first or only LAN interface)
- Manual configuration: ENABLED
10. Apply changes again.
11. Go to Services > Router Advertisements > LAN
- Router Advertisements: ASSISTED or STATELESS
- Advertise Default Gateway: ENABLED
- Use the DNS settings of the DHCPv6 server: ENABLED
- RA Sending: ENABLED
12. Optional: Configure DHCPv6 for LAN interface under Services > DHCPv6
- Range: ::1000 - ::1999
- Prefix Delegation Range: ::aa - ::ff
- Prefix Delegation Size: 64

These steps include a few obstacles I also identified for OPNsense:

- Router Advertisements seem to be set to "Router only". That means without explicitly enabling manual configuration to change this setting, stateless configuration will not work. There are still clients out there that can only use RA or DHCP6 to it is good to have this enabled.
- Enabling DHCP6 was somewhat difficult as it was actually required to also define a prefix delegation range. Without doing it, the DHCP6 server won't start.

That mostly describes my journey and took me some weekends to find out. Not sure if that would give some hints for some general improvements or fixes in OPNsense though...

[loredo]

After upgrading to version 18.7.9, this became worse again 
The workaround I described here no longer works, we're back to packet loss and serious delays for clients to access the network.

I've got no clue what to do for now... 

Every 180 seconds the router would stop to forward packages. After another 180 seconds, it will work for 3 minutes and then stop again... perfect loop and it seems obvious to me there is something wrong with managing the router solicitation packages.

[marjohn56]

There was an issue with the 'prefix delegation range' needing an entry, even if you are not using it, that has been resolved for 19.1.


Solicitation should not be needed every three minutes, so let's try and get closer to what is actually happening.


Firstly, check your dhcp logs, filter on dhcp6c and look for RENEW entries, you should see what is happening there, if all is clear there then we are looking  for something else.

[loredo]

Happy new year! 

The thing about the delegation range is something I also saw but didn't mention in my report above. Fortunately it is obvious as the UI would give an indication to add it. Not a big deal.

I did some more investigation on it and this is is what I can say:

1. Only enabling IPv6 on WAN but not on LAN would result in a stable connection: I can ping6 a target from the firewall directly and it would not be interrupted at all.
2. I would then enable Track Interface on LAN with manual configuration enabled. My Mac would then have IPv6 addresses assigned. However, ping6 from that client to an outside v6 destination would FAIL as the default route is missing. "Advertise Default Gateway" is enabled in the RA configuration. After a few more minutes, the default route also appears to the client. However, ping6 from the client would still fail while from the firewall, it continues to be okay. What disappeared is my IPv4 address on the firewall. In "Interfaces > Overview" it would say "DHCP - down". Clicking on "reload" would bring back IPv4 connectivity.

After this, connections seems stable for the moment (FW and client). I'll monitor the situation as I can't reproduce the issue right now :-/

[loredo]

Oh I should also mention that I have a VIP set on WAN to have a static address on the firewall for public services to listen. I left that VIP intact when disabling and re-enabling v6. Now I read about https://github.com/opnsense/core/issues/2189 and was wondering if this might be part of the issue