Hyperoptic IPv6 woes (no NDP advertisements received from upstream router)

[mjholgate]

Hey there,

Does anyone here use Opnsense with Hyperoptic in the UK?

IPv4 works great, but in the past I've had some trouble with IPv6 (see https://forum.opnsense.org/index.php?topic=19600.msg90614#msg90614). Despite this, IPv6 randomly started working one day and has been fine ever since...until now :-(.

The other day Hyper upgraded my package to a faster speed, and since then IPv6 seems to have gotten broken.

I've tried pinging the ISP's upstream gateway, but without success. Running tcpdump in another session, you can see that there are NDP solicitations being sent but I do not receive a reply advertisement from the ISP gateway:

$ ping6 'fe80::fa98:efff:fe73:892b'%igb0
...

$ tcpdump ip6
07:47:37.629209 IP6 fe80::xxxxxxx > ff02::1:ff73:892b: ICMP6, neighbor solicitation, who has fe80::fa98:efff:fe73:892b, length 32

The ISP supplied router seems to work fine with IPv6. Also if I connect my Mac directly to the ISP ethernet socket, then I can successfully ping6 the ISP upstream gateway, and I can see that neighbor advertisements are received correctly via wireshark.

Has anyone seen anything like this before? Are there any settings or tweaks that I could try making?

Note: As in my previous post, I'm still suspicious as I do not see any MLDv2 packets being sent from opnsense (and these are being sent from the Mac) - I'm not sure if this might have any effect on my ability to receive NDP replies?

Any help anyone can offer would be much appreciated as this is driving my nuts!
thanks
Matt.

[mjholgate]

Worth mentioning that I don't think this is a firewall issue as (a) I tried turning off the packet filtering and (b) I also did a capture using a managed switch with mirrored ports on the WAN port and didn't see any NDP advertisements either. I initially thought this was an ISP issue (as it worked before), but I don't understand why it is fine from my Mac.

Thanks!

[mjholgate]

Ok, I think I've finally gotten to the bottom of this. Here is the message I sent to Hyperoptic support - I'm hoping they can fix it at their end:

Hi there,

OK, I think I now understand what is going on here, and I have a temporary workaround. However, I think it possibly highlights an issue at the Hyperoptic end, and I would like to know what you think. Please could you pass this message onto one of your network engineers?

The problem seems to be with the way the Hyperoptic gateway interprets NDP neighbour advertisements from my third party router. Before the DHCPv6 handshake, the gateway sends an NS to my router, which my router responds to correctly. The NA reply has the 'R' flag set (which is correct, because it is a router and forwards ipv6). I noticed that the stock ZTE router sends its initial NA /without/ the 'R' flag set. I think because my router sets the flag, the Hyperoptic gateway discards my NA, and so cannot communicate with my router. Ping6's to the gateway from my router fail. The workaround is to disable DHCPv6 on my router, and switch to a static IPv6 configuration with just a link-local address while ping6'ing the gateway. This causes my router to send an NA *without* the R flag set, and the Hyperoptic gateway no longer ignores it. I can then successfully ping the gateway!

If I then switch the router back to DHCPv6, it can then successfully complete the DHCPv6 handshake and get allocated its /56. Because the NA is cached, things then continue to work. Is it by design that the Hyperoptic gateways behave in this way?

It would be good if it could be fixed, as the workaround isn't ideal as it requires manual intervention (and I'm also not sure if things will break again when the NA expires, or the Hyperoptic gateway is reset - which is guess what happened when my package was upgraded).

Many thanks
Matt.