Unbound periodically stops resolving some hostnames

Started by vrtigo1, March 05, 2024, 07:01:37 PM

Previous topic - Next topic
I started using opnsense about a month ago and like the title says, I've noticed on a handful of occasions the unbound resolver will periodically fail to resolve some hostnames.

There are almost universally sites that I visit infrequently. When I try using nslookup to manually query the unbound resolver running on opnsense for the hostname I'll get a 'server failed' error, but the opnsense resolver will continue to resolve other hostnames with no problem. After a few minutes the problem seems to go away and the previously unresolvable hostname works as expected.

This has happened with several different large websites, so I don't think this is an issue on the website's end.

Running 24.1.1 in a pretty vanilla configuration on an N100 micro PC.

After enabling unbound logging, I initially saw logs like this:

2024-03-04T21:23:04-05:00 Error unbound [66128:3] error: SERVFAIL <link.ablink.hardrockgames.com. AAAA IN>: exceeded the maximum number of sends

2024-03-04T21:23:04-05:00 Error unbound [66128:0] error: SERVFAIL <link.ablink.hardrockgames.com. A IN>: exceeded the maximum number of sends

2024-03-04T21:23:04-05:00 Error unbound [66128:1] error: SERVFAIL <link.ablink.hardrockgames.com. AAAA IN>: exceeded the maximum number of sends

2024-03-04T21:23:04-05:00 Error unbound [66128:2] error: SERVFAIL <link.ablink.hardrockgames.com. A IN>: exceeded the maximum number of sends

2024-03-04T21:22:54-05:00 Error unbound [66128:1] error: SERVFAIL <catalog.gamepass.com. A IN>: exceeded the maximum number of sends

2024-03-04T21:22:47-05:00 Error unbound [66128:3] error: SERVFAIL <push.prod.netflix.com. A IN>: exceeded the maximum number of sends

2024-03-04T21:22:44-05:00 Error unbound [66128:3] error: SERVFAIL <mn04-lobby-gate.mattel163.com. HTTPS IN>: exceeded the maximum number of sends

2024-03-04T21:22:44-05:00 Error unbound [66128:0] error: SERVFAIL <mn04-lobby-gate.mattel163.com. A IN>: exceeded the maximum number of sends

2024-03-04T21:22:43-05:00 Error unbound [66128:3] error: SERVFAIL <tc-log.mattel163.com. A IN>: exceeded the maximum number of sends


After cranking the verbosity up, I was seeing things like this:

2024-03-05T12:45:51-05:00 Error unbound [6767:1] error: udp connect failed: No route to host for 2001:502:7094::30 port 53 (len 28)

It looks like unbound is trying to use IPv6 for some reason, even though IPv6 is disabled on all my opnsense interfaces.

Any suggestions on how I can troubleshoot this issue?

I don't know if plays a part but check: System > Settings > General > Networking | " Prefer to use IPv4 even if IPv6 is available "
maybe ?


Quote from: cookiemonster on March 05, 2024, 10:36:58 PM
I don't know if plays a part but check: System > Settings > General > Networking | " Prefer to use IPv4 even if IPv6 is available "
maybe ?

Unfortunately I'm still seeing the same behavior with this setting enabled.  Any other thoughts?

I'm on OPN version:
OPNsense 23.7.12-amd64
FreeBSD 13.2-RELEASE-p7
OpenSSL 1.1.1w

I don't use ipv6. I've disabled it in most places I've seen it. That's one of those I suggested but I don't see any traces of Unbound trying to use it. Maybe on your version of OPN Unbound behaves differently.
You should always in your posts include your versions (done) and setup so people can help. Otherwise it is too much to guess. pihole, AdguardHome involved at all? Other devices in the network?

Is IPv6 disabled on your WAN?
What do you have for your DNS settings on System: Settings: General?
Do you have any entries under Services: Unbound DNS: Query Forwarding or Services: Unbound DNS: DNS over TLS?


As a side note, enabling IPv6 just for unbound can be handy as resolvers return both v4 and v6 records.  It's the only v6 traffic I currently have on my network.

Quote from: CJ on March 07, 2024, 01:11:11 PM
Is IPv6 disabled on your WAN?
What do you have for your DNS settings on System: Settings: General?
Do you have any entries under Services: Unbound DNS: Query Forwarding or Services: Unbound DNS: DNS over TLS?


As a side note, enabling IPv6 just for unbound can be handy as resolvers return both v4 and v6 records.  It's the only v6 traffic I currently have on my network.

Yes IPv6 is completely disabled on all interfaces.

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

Nothing under Unbound DNS > Query Forwarding or DNS over TLS

Quote from: vrtigo1 on March 07, 2024, 08:36:06 PM
Quote from: CJ on March 07, 2024, 01:11:11 PM
Is IPv6 disabled on your WAN?
What do you have for your DNS settings on System: Settings: General?
Do you have any entries under Services: Unbound DNS: Query Forwarding or Services: Unbound DNS: DNS over TLS?


As a side note, enabling IPv6 just for unbound can be handy as resolvers return both v4 and v6 records.  It's the only v6 traffic I currently have on my network.

Yes IPv6 is completely disabled on all interfaces.

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

Nothing under Unbound DNS > Query Forwarding or DNS over TLS

What do you have under DNS server options on the General page?

Quote from: CJ on March 08, 2024, 03:51:45 PM
What do you have under DNS server options on the General page?

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

March 12, 2024, 10:15:32 PM #9 Last Edit: March 13, 2024, 12:03:15 PM by Azokul
Hi,
I have had the same problems since a few updates ago.
I'm on  OPNsense 24.1.3_1-amd64

IPv6 Is disabled overall, I'm using 8.8.8.8 or 1.1.1.1 as default DNS on opnsense, with no override on LANs.
In unbound I don't have DNSSEC and I don't have query forwarding ON.
Every now and then I get SERVFAIL for exceeded maximum requests, I have up to 8000 contemporary requests at specific times of the day.
With dnsqmasq I have no problems

Quote from: vrtigo1 on March 12, 2024, 02:27:10 AM
Quote from: CJ on March 08, 2024, 03:51:45 PM
What do you have under DNS server options on the General page?

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

Not DNS servers.  DNS server options.  The section below where the DNS servers are entered.

Quote from: Azokul on March 12, 2024, 10:15:32 PM
Hi,
I have had the same problems since a few updates ago.
I'm on  OPNsense 24.1.3_1-amd64

IPv6 Is disabled overall, I'm using 8.8.8.8 or 1.1.1.1 as default DNS on opnsense, with no override on LANs.
In unbound I don't have DNSSEC and I don't have query forwarding ON.
Every now and then I get SERVFAIL for exceeded maximum requests, I have up to 8000 contemporary requests at specific times of the day.
With dnsqmasq I have no problems

You have a different issue.  Please start a new thread.

Quote from: CJ on March 13, 2024, 01:09:19 PM
Quote from: vrtigo1 on March 12, 2024, 02:27:10 AM
Quote from: CJ on March 08, 2024, 03:51:45 PM
What do you have under DNS server options on the General page?

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

Not DNS servers.  DNS server options.  The section below where the DNS servers are entered.

Nothing checked there.

March 14, 2024, 09:26:44 PM #13 Last Edit: March 15, 2024, 01:17:05 AM by Azokul
Is your unbound also serving WAN and maybe you got ACL overrides?
Just as a personal reference after I disabled WAN requests , my problems went away (I also increased number of queries per thread)

Quote from: vrtigo1 on March 13, 2024, 05:57:14 PM
Quote from: CJ on March 13, 2024, 01:09:19 PM
Quote from: vrtigo1 on March 12, 2024, 02:27:10 AM
Quote from: CJ on March 08, 2024, 03:51:45 PM
What do you have under DNS server options on the General page?

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

Not DNS servers.  DNS server options.  The section below where the DNS servers are entered.

Nothing checked there.

Quote from: vrtigo1 on March 13, 2024, 05:57:14 PM
Quote from: CJ on March 13, 2024, 01:09:19 PM
Quote from: vrtigo1 on March 12, 2024, 02:27:10 AM
Quote from: CJ on March 08, 2024, 03:51:45 PM
What do you have under DNS server options on the General page?

Under system > settings > general > DNS servers I have 1.1.1.1 and 8.8.8.8.

Not DNS servers.  DNS server options.  The section below where the DNS servers are entered.

Nothing checked there.

That's what I figured.  Right now you have Unbound operating in resolve mode which hits the root servers.  I assume you want it to be working in forwarding mode and using 1.1.1.1 and 8.8.8.8.

On Services: Unbound DNS: Query Forwarding check the Use System Nameservers checkbox.  If you prefer to use DoT, you can set that instead but then I'd recommend removing the entries from the General tab.