IPv6 problems with APU2 (possibly related to multicast MLDv2).

[mjholgate]

Hi there,

I'm a newbie to OPNsense, and have installed 20.7.3 on my PCengines APU2 box. I'm really impressed with how powerful and easy to use it is! Nice work and thanks :-).

IPv4 is working great, but i'm having some issues with IPv6. It works fine with the ISP's stock (ZTE) router, but I can't get it to work with OPNsense.

My ISP is Hyperoptic (UK) who run ethernet to the premises and then support native IPv6 - allocating a /56 prefix via DHCPv6 and configuring the router's own address using SLAAC.

After a lot of experimentation, I ended up using a managed switch with port monitoring to intercept the traffic to compare the two routers. Please see the screenshots below:

ISP router (ZTE):
http://www.holgate.org.uk/working-isp-router.png

Opensense:
http://www.holgate.org.uk/not-working-opnsense.png

The main thing i noticed was that OPNsense doesn't appear to be sending MLDv2 announcements.

Ultimately I seem to be unable to ping6 the ISP's router, and using the ndp command, I can see that I'm unable to resolve the MAC address. I was wondering if this might be due to the lack of Multicast Listener Report packets?

I did a bit more experimentation using tcpdump and mtest and noticed that these announcements were being sent fine on igb1 (LAN), but not on igb0 (WAN).

I tried lots of different config changes (including completely disabling packet filtering), but to no avail. Occasionally a packet would seem to slip through and be visible via tcpdump, but I've no idea what changed to allow this.

I just wondered if anyone has any insights into this problem as it's driving me crazy and not sure what else to try!

Many thanks in advance,
Matt.

[gary201]

I have a similar symptom, though I have no idea whether it's related to what you are seeing.  I literally have to reboot OpnSense every morning to keep ipv6 working.  If I don't, then sometime during the night, ipv6 stops working (can't ping anything from OpnSense out to the wan).  Restarting services (11 on the login menu) doesn't work.  It actually has to be a reboot. 

[gary201]

How does posting issues here get on some sort of bug list for fixing?  Are we just venting here or do issues posted here actually get worked on?

[sToRmInG]

Do you have to use PPPoE to connect to your ISP?

[marjohn56]

The main thing i noticed was that OPNsense doesn't appear to be sending MLDv2 announcements.

Just looked at my test router, plugged in directly to my PC Wireshark port ( I have one specifically as it's connected to my switch for port mirroring and those messages are definitely there. It may be an issue with the DUID; possibly HyperOptic are looking for a specific device. When I was with Sky some years back we used to have an issue that if the DUID changed then their server would just ignore it, you either had to find the DUID of the Sky router and paste that into pfSense, yes before the days of OpnSense or disconnect the phone line completely and wait for 30 to 45 minutes for the BNG to forget about you and then you might get an address when you reconnected.


Anyway, try grabbing the DUID from the supplied router, you're almost there, if you look at the dhcpv6 packets from their router and expand the dhcpv6 packet you'll see the DUID value, copy that and paste it into the DHCP Unique Identifier box in Interfaces->Settings. You'll need to add colons between the pairs, so if the value you get from wireshark is 0001000126da282d000ec4d28141
then you need to enter into Opnsense like this
00:01:00:01:26:da:28:2d:00:0e:c4:d2:81:41

It's a bit like spoofing the MAC address.Try it and see if it makes any difference.

[gary201]

I've been doing some more testing on/off and what I have found is that IPv6 works only for a few minutes after a reboot, then it just stops working.  My ISP is Greenlight Networks (a residential fiber provider in western NY State).  When it stops working, reboot the router and it works.  Then after some amount of time less than 20 minutes it stops working.

[marjohn56]

How does posting issues here get on some sort of bug list for fixing?  Are we just venting here or do issues posted here actually get worked on?

They get read, most of the time it's not a bug but just a little advice that's needed. If it's a real bug then it needs posting on Github, but you need to make sure it's a real bug otherwise the wrath of the devs will fall upon you.

[marjohn56]

I've been doing some more testing on/off and what I have found is that IPv6 works only for a few minutes after a reboot, then it just stops working.  My ISP is Greenlight Networks (a residential fiber provider in western NY State).  When it stops working, reboot the router and it works.  Then after some amount of time less than 20 minutes it stops working.

Does it work fine with their router?

[gary201]

I'm not sure I understand what you mean.  If I reboot OpnSense, ipv6 works, for some amount of time less than 20 minutes (only because I haven't sat around testing every few min).  After that some amount of time, it fails.  By working and failing, I'm talking about pinging google (from opnsense, ping6 google.com; from windows ping -6 google.com).

[gary201]

It appears that the point of failure is 4 minutes after OpnSense is rebooted. 

[marjohn56]

I assume that your ISP supplied a router, is it OK on their router. Saying it stops working doesn't really help either. What diagnostics have you carried out?


Have you tried pinging from Opnsense, Interfaces->Diagnostics->Ping ?


Have you set up WAN6 gateway monitor with a GUA address, does that continue to work?


What do the logs say?

[mjholgate]

Thanks everyone for the help, especially @marjohn56 for the detailed suggestion re the DUID!

Weirdly, the next day, everything seemed to start working fine without any real changes.

I'm wondering if perhaps the stock router's previous lease eventually expired and unblocked the connection? NB. I had previously done some experiments with MAC spoofing, but it didn't really help, and eventually it seemed to work without it (and without spoofing the DUID). So who knows!

I also did see a problem early on where the IPv6 connection would sometimes lose its route overnight (much like other people have mentioned on this thread!). I changed the settings to not allocate an IP to the router itself via DHCPv6 (just a /56 prefix), and that /may/ have fixed it. I still seem to have an IPv6 address on the router's WAN port, but that must have been set via SLAAC (and reading other threads a mixed approach of DHCPv6/SLAAC seems to what Hyperoptic require).

So, everything is now working great, but I'm keeping an eye on it and have setup monitoring via smokeping on an internal host to see if there are any further issues. 🤞🏻🤞🏻🤞🏻🤞🏻

If others are seeing the connection drops, try "Request only an IPv6 prefix" as I think that may have been what fixed it for me.

thanks again everyone,
Matt.

[mjholgate]

I should add, at first I didn't seem to manage to get IPv6 working on the router itself (but it is working now, weirdly).

If this happens, set "Prefer IPv4 over IPv6" in Settings > General, otherwise I found the router was unable to check for firmware updates.

[marjohn56]

Well if it works that's fine, confirm it by using https://ipv6-test.com/.


I also recommend that you save and use the existing DUID if it's working, under the DUID box is an option called 'Insert existing DUID' click on that then save/apply. This means dhcp6c will always use the same DUID, otherwise when you reboot and if your /var /tmp are RAM folders it will generate a new one, and you may lose ipv6 again. Also select "Prevent release", this will stop dhcp6c sending a release of the existing lease to the ISP.

[mjholgate]

Thanks @marjohn56, it's definitely working with the IPv6 test site.

Also, I've saved the DUID and disabled release - good call on that.

Thanks again for all your help
Matt.