Messages

Ah, yes, you know it's fun when it has two prefixes before -DNS .

Thank you.

I'm familiar with the firewall rules to force DoT. My concern right now is that I'm running a half-ass configured split-brain DNS and that is why I can't get any activity on port 853

[Current Configuration:]

I have not been able to verify my Unbound DNS over TLS is configured correctly. I've followed every tutorial to the letter, and every test I've run, port 853 is nowhere to be found.

Current Configuration:

• General DNS settings: All fields left blank
• DNS over TLS: Configured with Cloudflare's DNS over TLS servers

Background Context:
My local domain happens to be a legitimate domain that I own and have registered through Cloudflare. This configuration has been in place since my initial OPNsense installation, and I haven't implemented any advanced configurations like DNS overrides.

Question:
If I were to create DNS overrides for each VM in my network, would this ensure that local VM resolution uses DNS over TLS when accessed from my workstation?
I'm trying to understand if my current setup is preventing proper DNS over TLS functionality, and whether implementing local DNS overrides would resolve the issue.
Any guidance on troubleshooting DNS over TLS verification or recommendations for proper local domain handling would be greatly appreciated.

The /60 appears to only be used to satisfy some UI validation:
https://forum.netgate.com/post/962817 (post from deet)
The workaround in these threads is to request up to 8 /64 and manually assign them to interfaces...

The original thread started 5 years ago. AT&T has not "fixed" any of this yet???
This is definitely not following deployment recommendations...

My ISP is late deploying IPv6 (imminent according to some of their feeds) but at least they are doing it by the book.

Thanks for checking. There is probably not going to be some silver bullet, and I'm going to have to write some scripts to resolve the issue, which I'm sure will be a case of trial & error that I don't have time for 🙃

OK, that's great information, man, I'm going to have to copy/paste this in my workspace. My company is scattered throughout the globe, and there's zero humans on my LAN, aside from me, so I was considering just putting it on WG0 (wireguard) because I'm hosting a fairly substantial VPN server. They're the ones that need the protection of a NGFW the most.

No need to apologize; the topic is essentially about AT&T.

https://www.reddit.com/r/opnsense/comments/17mw7ys/att_ipv6_and_multilan_oh_my/
https://github.com/lilchancep/att-pfsense-ipv6
https://forum.netgate.com/topic/153288/multiple-ipv6-prefix-delegation-over-at-t-residential-gateway-for-pfsense-2-4-5/11
https://www.perplexity.ai/search/cedf07dc-1fe2-44e3-b896-b61a13122db1 (Aggregated a ton of articles/posts)

Thanks- this was enlightening.

So apparently AT&T responds to a request for a /60 but they don't actually delegate the full range.  They only hand out 8 prefixes, requiring users to manually override DHCP settings with custom config files.

(It seems that also AT&T gateways can't be put into bridge mode, leading users to bypass them with special SFP modules that emulate an ONT)

This sounds like a difficult ISP to deal with :-/

Yes, except that it's unclear if the custom config file solution "just works." I believe one of the users referenced in those links mentioned that it was not an iron clad solution; which is also the feedback I was given.

Also, the BGW320 does not have a bridge mode, nor does the BGW-325, or whatever the third one is. They used to assign standard gatweways that could go in true bridge mode, not cause Double NAT or mess with IPv6. Unfortunately, for AT&T, those gateways were no good at harvesting their users' data. 90% of the compute in that gateway goes to telemetry, not NAT/Firewall/Routing/WiFi. No conspiracy either, if a user logs into their account settings and spends enough time, they will find that they were automatically opted-in for very invasive data collection and authorized AT&T to monetize that data.

You dont require dhcpv6 on the client side to have IPv6 connectivity, what you mostly use right now is radvd for RA to your LAN. Its separate from ISC.

So the combination of radvd + isc dhcpv6 = dnsmasqs capabilities (mostly).

With RA flags I mean this:

https://github.com/opnsense/docs/commit/a84eb16879ccd6ff21cd067856ce973bd8e30ba9

As you can see, different combinations will change RA and DHCPv6 behavior.

Correct. I depend on radvd for Router Advertisements, which enables SLAAC for my LAN clients. I'm not looking to implement stateful DHCPv6—I just need the existing IPv6 connectivity to stop disappearing.

For example, several GitHub Actions workflows failed this morning due to IPv6 connectivity loss. While it wasn't critical on a Sunday, this issue occurs unpredictably throughout the week. Unfortunately, disabling IPv6 isn't an option, as the majority of our CI/CD infrastructure depends on it.

Given that my WAN-side prefix delegation from AT&T remains stable, and I've observed multiple radvd errors over the last year, the issue must be on LAN side with radvd.

I'm reviewing the updated documentation and will migrate to dnsmasq tonight, pushing logs to a remote database for better observability.

It depends on why and where the connection stops to work.

Essentially DHCPv6 on the LAN side should not be required for IPV6 to work. All you need is Router Advertisements (RA) which advertises the default route and sets a flag that clients do SLAAC.

You should look into if the clients forget the default route or deprecate their SLAAC addresses prematurely, which will make IPv6 routing fail until they got a new one.

If you plan on using dnsmasq, it just tracks the prefix configured on interfaces and can do RA and optionally DHCPv6 for the clients.

I imagine most connectivity issues to be with the dhcpv6 client and rtsold on the WAN interface, which has nothing to do with dnsmasq. If the firewall itsel intermittently looses the default route to AT&T, using dnsmasq wont solve anything.

We're on the same page. I'll respond in order:

-Having it set to Track WAN Interface, with RA setup, has given me the most success, but it doesn't stop IPv6 on LAN from going down for hours or days, seemingly randomly.

-I'm not sure what you mean about setting a flag. This is outside my understanding of the platform unless you're referencing tunables?

-I do need to block off a chunk of time to do a deep dive through the logs. I have been meaning to set up something like Graylog. I'd be lying if I said I didn't get lost in the robust web UI of OPNsense.

-Regarding plans to use Dnsmasq and how it tracks the prefix configured on interfaces and can do RA, this is more or less what I was getting at in my original post. It appears to be more intuitive than ISC, which has been a nightmare for me, requiring me to go to four or five different pages to get IPv6 running.

-Regarding the WAN interface, this is something I have tracked intently, and it never loses IPv6 connectivity.

hanks, and sorry to divert the topic.

I don't understand what the issue with regard to the AT&T prefix delegation is after reading this thread and the one you linked in your OP.  What do you mean by this?

The problem isn't that they delegate a /60 prefix, but rather the fragile and unorthodox way they expect their gateway to request it

Thanks, and sorry to divert the topic.  It just want to be sure I understand because AT&T is one of the ISPs I would consider moving to eventually.

No need to apologize; the topic is essentially about AT&T.

https://www.reddit.com/r/opnsense/comments/17mw7ys/att_ipv6_and_multilan_oh_my/
https://github.com/lilchancep/att-pfsense-ipv6
https://forum.netgate.com/topic/153288/multiple-ipv6-prefix-delegation-over-at-t-residential-gateway-for-pfsense-2-4-5/11
https://www.perplexity.ai/search/cedf07dc-1fe2-44e3-b896-b61a13122db1 (Aggregated a ton of articles/posts)

Hello,

I think you are mixing up different IPv6 concepts, read through this article:

https://docs.opnsense.org/manual/ipv6.html#technical-background

(Dnsmasq is a dhcpv4 and 6 and RA server, not a client)

Hello Cedrik,

I believe there may be a misunderstanding. I understand that Dnsmasq services the LAN, and is not a client. My question is specifically about the LAN-side DHCPv6 server implementation.

OPNsense successfully maintains the /60 prefix delegation from AT&T on the WAN side. The issue is that LAN clients experience intermittent IPv6 connectivity drops. Since I've been using ISC DHCP for the LAN-side DHCPv6 server, I was asking whether Dnsmasq's DHCPv6 server implementation might handle the delegated prefixes more reliably for LAN clients.

Is there any evidence that one DHCPv6 server implementation works better than another with AT&T's delegated prefix

Since bypassing AT&T's BGW320 gateway back in January 2023, I have happily been running OPNsense with one caveat -- the necessary evil that is IPv6. It's been a constant cat-and-mouse game, and as I've posted before, the root of that evil is AT&T's proprietary and non-standard approach to IPv6.

The problem isn't that they delegate a /60 prefix, but rather the fragile and unorthodox way they expect their gateway to request it, which remains a constant struggle for OPNsense & PFsense users (I believe Unifi, as well). I have only used ISC & Unbound, but from what I've read so far, the consensus is that Dnsmasq DHCP has superior DHCPv6 support.

Would making the move to Dnsmasq be the solution to my IPv6 issues? Thank you.

https://docs.opnsense.org/troubleshooting/performance.html#receive-side-scaling

Wow. Interesting..........

So there's three options: RSS, Unpin CPU, & push logs to remote database, correct?

Hi,

Good news regarding multicore support. We have released a test binary. Kindly contact the support team via the "Have Feedback" option located in the bottom right corner of the UI if you wish to try it out.

Thank you, I will do this now.

Has it made a difference in your use-case?

I just set it up, per the instructions that were emailed to me, but there seem to be errors. I forwarded the logs to the ZenArmor team.

When they get back to me tomorrow I will troubleshoot and report back.

Similar boat for me. Tons of issues with IPv4 connectivity as well. Hopefully they can get some of this resolved :)

I don't have the logs available right now, because I'm on my phone, but the multi-threaded throughput was actually lower than single-threaded.

Unfortunately, I had to uninstall the service until they can polish this new feature. My OPNsense server's two CPUs have the highest rates single-threaded performance available for the PowerEdge R730.

I was not using the Pin feature.

Off the top of my head I am not sure what RSS is.

Hi,

Good news regarding multicore support. We have released a test binary. Kindly contact the support team via the "Have Feedback" option located in the bottom right corner of the UI if you wish to try it out.

Thank you, I will do this now.

Has it made a difference in your use-case?

I just set it up, per the instructions that were emailed to me, but there seem to be errors. I forwarded the logs to the ZenArmor team.

When they get back to me tomorrow I will troubleshoot and report back.