IPv6 DNS issues in 23.1.8?

[Zoltrix]

Hi,

A few weeks ago I migrated from pfSense to OPNsense 23.1.7. It was running without issue. Yesterday I updated to 23.1.8, and a lot of clients on my network started to be assigned an IPv6 DNS server (in addition to the IPv4 one). This seemed to cause the network to go a bit haywire, not sure why? Currently using Unbound on OPNsense for the DNS server.

I completely disabled IPv6, the clients stopped getting the IPv6 DNS server, and everything seems smooth again.

Cheers,
Zolt

[sbellon]

I have the same issue.

When visiting https://ipv6-test.com/ previously I constantly got 18/20 (missing two points are due to ICMP6), but with 23.1.8 it dropped down to 14 or even lower because the DNS part became very flaky.

On Windows clients when doing ipconfig /all I realized that with 23.1.8 a DNS IPv6 is handed out, which was not the case with 23.1.7_3.

On GNU/Linux and Android clients however, also with 23.1.7_3 the DNS IPv6 is handed out.

Most likely this is due to the different ways of handing out IPv6 to the OS (DHCPv6 vs. SLAAC, etc.).

I have not debugged any further but also reverted to 23.1.7_3 because that makes my network much more stable.

BTW: I'm using Dnsmasq on OPNsense as DNS server which forwards to a Pi-hole. Both, OPNsense and Pi-hole can be DNS-queried using their IPv6 ULA successfully when trying with dig/host/nslookup.

[franco]

I completely disabled IPv6, the clients stopped getting the IPv6 DNS server, and everything seems smooth again.

Sounds like IPv6 started working after update. Disable if not required and that's it. The thing with defaults is that you have them set so they could be working or not... DHCPv6/Track6 are the default for WAN/LAN.

@sbellon how is that "the same issue" when you report the reverse of what OP said?


Cheers,
Franco

[sbellon]

Now I'm confused ...

A few weeks ago I migrated from pfSense to OPNsense 23.1.7.

So did I.

It was running without issue.

So was it for me.

Yesterday I updated to 23.1.8, and a lot of clients on my network started to be assigned an IPv6 DNS server (in addition to the IPv4 one).

Exactly the same in my setup (for the Windows clients).

This seemed to cause the network to go a bit haywire, not sure why?

Same here.

Currently using Unbound on OPNsense for the DNS server.

Ok, I'm using Dnsmasq instead.

Sorry, but why do you think I reported the opposite?

[franco]

A few weeks ago I migrated from pfSense to OPNsense 23.1.7.

So did I.

Ok sorry I cannot keep track of your on and off relationship with OPNsense.


Cheers,
Franco

[sbellon]

I think we are confusing each other now.

I do NOT have an on and off relationship with OPNsense. I am consistently using OPNsense since 2021 when I migrated from Ubiquiti Security Gateway.

My "So did I" was misleading ... I was just pointing out what I did w.r.t. 23.1.7 and what has now changed with 23.1.8 ... I wanted to express "a few weeks ago I upgraded to OPNsense 23.1.7 and everything was fine with 23.1.7 like for the OP", not that I migrated to OPNsense 23.1.7 from pfsense (as the OP).

[Zoltrix]

One of the reasons I migrated, was because the forums over at pfSense were borderline abusive, and not very supportive. Hoping to have a different experience over here.

Thank you for the comments franco, but why a default installation would cause network issues when IPv6 starts working, seems a little strange to me?

[franco]

@sbellon

Apologies for misreading then. I'd always wish for more concise reports qmbiguity is the enemy of community support.


@Zoltrix

It's not unusual that the IPv6 defaults can be a factor of connectivity issues with certain ISPs. We have such reports regularly, mostly pertaining to problems with firmware updates. I don't think IPv6 connectivity issues reach the internal clients too often if such a fundamental issue exists. Also it has been known that some ISPs meddle with DNS resolution to the point where it looks like it's broken.

As such it's impossible to give a generalised assessment of what the issue is. Thus the rule of thumb is if you don't want or use IPv6 clear the IPv6 modes of your interfaces.


Cheers,
Franco

[franco]

PS: as for IPv6 issues an overview of WAN and LAN IPv6 modes on your install helps as well as any relevant logs. If DHCPv6 (dhcp6c) is involved on the WAN it would make sense to enable the debug mode for IPv6 and reboot and gather the actual connection info from the client.

[sbellon]

@sbellon

Apologies for misreading then. I'd always wish for more concise reports qmbiguity is the enemy of community support.

To be honest, I didn't even realize that the OP was coming from pfsense, my brain just saw OPNsense, the same version number and "the same" (or similar) issue than mine.

Back to the actual issue: Would it help (or would you want that at all?) if I sent you my configuration via some private channels?

Today even my wife (using iOS) admitted that yesterday she turned off Wifi and used 5G in the house as "everything was unbelievable slow". Since downgrading to 23.1.7_3 everything is back to normal.

So, there IS something going on with IPv6 from 23.1.7_3 to 23.1.8 ... not saying it is the fault of OPNsense, perhaps its just the broken configuration that worked in the past and now needs proper adjustments.

[Cyberturtle]

With 23.1.8 a few things changed for IPv6 and how to handle it. So when you guys didn't have IPv6 with 23.1.7_3 at all and after 23.1.8 IPv6 came in I'd like to say it's showing that IPv6 is on the right way in OPNsense.
When your clients receive a DNS IPv6 server now and if this server is unreliable this can lead into issue. Of course IPv6 heavily relies on properly configured MTU and MSS. For example I have to setup MTU and MSS to 1492 as I'm using PPPoE.
Which IPv6 DNS server are your clients getting? Is it the OPNsense IPv6 or another one? How is your Router Advertisments service configured (Unmanaged, Assisted etc.)? Or do you have configured a traffic shaper without considering IPv6?

Edit: Apple devices, for example, first use DNS IPv6 if present and if it fails it will use DNS IPv4 as required by RFC. This can slow down connections sometimes. As france already mentioned, please provide us an overview of your IPv6 config.

[sbellon]

I'll try to do a write-up what and how I configured my network regarding IPv6:

System / Settings / General
- DNS servers: IPv4 of Pi-hole
- NOT SET: Prefer to use IPv4 even if IPv6 is available
- NOT SET: Allow DNS server list to be overridden by DHCP/PPP on WAN
- NOT SET: Do not use the local DNS service as a nameserver for this system

Interfaces / WAN:
- IPv4 is PPPoE
- IPv6 is DHCPv6
- MTU is left empty but says "Calculated PPP MTU: 1492" (that's why I assume I can leave it empty)
- Request only an IPv6 prefix
- Prefix delegation size 56
- Send IPv6 prefix hint
- Use IPv4 connectivity

Interfaces / LAN:
- IPv4: Static IPv4
- IPv6: Track Interface
- MTU is left empty
- Track interface: WAN
- IPv6 Prefix ID: 0x0
- Allow manual adjustment of DHCPv6 and Router Advertisements

Interfaces / Settings:
- IPv6 DHCP: Prevent Release

Interfaces / Virtual IPs / Settings:
- IP Alias LAN for ULA of OPNsense

Services / DHCPv4:
- DNS servers: IPv4 of OPNsense

Services / DHCPv6:
- DNS server: IPv6 ULA of OPNsense (virtual IP alias)

Services / Dnsmasq DNS / Settings:
- Enabled
- Register DHCP leases
- Register DHCP static mappings
- Resolve DHCP mappings first
- Query DNS servers sequentially
- Require domain
- Do not forward private reverse lookups

Services / Router Advertisements / LAN:
- Stateless
- Use the DNS configuration of the DHCPv6 server
- NOT SET: Do not send any DNS configuration to clients


Again, this works perfectly fine with 23.1.7_3 and I do get a score of 18/20 on https://ipv6-test.com/, and if I even enable an ICMP6 rule in the firewall, I get a full 20/20.

After upgrading to 23.1.8, DNS resolution from various clients became slow (most likely running into various timeouts) up to completely unreliable. I noticed that on the Windows client, the IPv6 ULA of the OPNsense is handed out as DNS server to the clients, which is not the case with 23.1.7_3. On GNU/Linux I get the IPv4 and the IPv6 ULA of the OPNsense in /etc/resolv.conf with 23.1.7_3 and with 23.1.8 (same on Android). Have not checked with iOS, but as already mentioned, my wife's experience was reduced to an extend that she switched to 5G instead of Wifi...

I hope I did not forget anything important, otherwise please ask.

[Cyberturtle]

Thanks for the overview.
Is your Pi-hole able to resolve AAAA records and can your Pi-hole also provide an IPv6 address? Under settings general you can then enter the IPv6 address as DNS and assign it to DHCP6 gateway. The IPv4 DNS server can be assigned to PPPoE.
Try setting a MSS value of 1492 at WAN interface. Furthermore I would enable parallel DNS queries by disabling ,,Query DNS servers sequentially". If it doesn't work you could check ,,Do not send any DNS configuration to clients".

Hope it will help.

[sbellon]

Is your Pi-hole able to resolve AAAA records and can your Pi-hole also provide an IPv6 address?

Yes, to both.

Under settings general you can then enter the IPv6 address as DNS and assign it to DHCP6 gateway. The IPv4 DNS server can be assigned to PPPoE.

This is not possible, as it results in an error message in the OPNsense GUI if I try to configure it:

Code Select Expand


The following input errors were detected:

You can not assign a gateway to DNS server "192.168.1.9" which is on a directly connected network.
You can not assign a gateway to DNS server "fe80::f7:b7ff:fe24:55ca" which is on a directly connected network.

And I think this makes sense, if I assign a DNS server to a gateway, then the meaning is "use this gateway to reach the DNS server", isn't it? But in my case the Pi-hole is on the local network, not outside.

I added the ULA IPv6 of the Pi-hole to System / Settings / General as a second entry in addition to the IPv4. I can verify that it gets written into /etc/resolv.conf of the OPNsense and I can query via its IPv6 and also AAAA type from within the CLI of the OPNsense.

Try setting a MSS value of 1492 at WAN interface.

So, you are saying that I should explicitly configure 1492 for both, MTU and MSS?

Furthermore I would enable parallel DNS queries by disabling ,,Query DNS servers sequentially".

I had this disabled because it shouldn't have made any difference if I only have one DNS server configured, should it? Now with the IPv4 and the IPv6 it would perhaps make sense to use the faster of the two (although as it is the same device, this would be for IPv4/IPv6 redundancy only and not performance).

If it doesn't work you could check ,,Do not send any DNS configuration to clients".

But that's not what I want. I do want clients to configure their IPv4 and IPv6 DNS server via DHCPv6 and DHCPv6 (which works with 23.1.7_3).

[Cyberturtle]

Configuring at least MSS clamping with a value of 1492 should be done as IPv6 does not fragment packets like IPv4 does. Only the source is allowed to fragment but has to know which value has to be considered. I have also set MTU even it should be detected automatically.

I understand you that way that the IPv6 DNS server showed up initially with 23.1.8. So that's why I suggested disabling sending IPv6 DNS server in the router advertisement. So your clients are only receiving the IPv4 one. Just for testing and trial and error.

I don't have any knowledge about Pi-hole config so someone else can help hopefully.

Did dnsmasq receive an update with 23.1.8?

Edit: Can you please try test-ipv6.com? This site was able to detect MTU problems in the past.