OPNsense Forum

Archive => 21.1 Legacy Series => Topic started by: Demus4202 on June 19, 2021, 10:57:42 pm

Title: Unbound DNS Locking Up
Post by: Demus4202 on June 19, 2021, 10:57:42 pm
Hello everyone, I just updated to version 21.1.7_1, and now am having what appears to be issues with Unbound crashing every few hours seemingly at random.

As far as I can tell, when it happens, DNS lookups fail but I can still ping out via IP address from a terminal. The only way I have found to solve it is by restarting Unbound or rebooting OPNsense all together.

I've looked in the logs that I know about and don't see anything catastrophic. Also checked the forums and tried several things listed for older versions of OPNSense, all with no luck.

Also, not sure if it is of any help, I have Unbound set up to be forwarding queries to NextDNS over TLS.

Thanks in advance for any help!
Title: Re: Unbound DNS Locking Up
Post by: Msan on June 19, 2021, 11:50:33 pm
I have a similar issue.. but I am still on 21.1.6
unbound seems to lock up and doesnt respond to dns queries. but in my case it "only" happens every few days. I was hoping that 21.1.7 would fix this...
Title: Re: Unbound DNS Locking Up
Post by: dinguz on June 20, 2021, 02:16:52 pm

Also, not sure if it is of any help, I have Unbound set up to be forwarding queries to NextDNS over TLS.


What happens when you disable DoT and/or forwarding?
In my setup, forwarding works, but unbound stops working as soon as I enable DNS over TLS.
Title: Re: Unbound DNS Locking Up
Post by: Msan on June 21, 2021, 11:07:27 pm

Also, not sure if it is of any help, I have Unbound set up to be forwarding queries to NextDNS over TLS.


What happens when you disable DoT and/or forwarding?
In my setup, forwarding works, but unbound stops working as soon as I enable DNS over TLS.

It just happened to me again.. On 21.1.7_1 now.. And I have DNS over TLS disabled..
Restarting unbound seems to fix it for a while..
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on June 22, 2021, 01:54:48 am
I haven't touched anything on my config and it seems to be stabilized. It seems really weird that it was doing it for several days straight and now its been solid for a day or so.

I still have DoT running as well. I built a backup Pi-Hole just in case I need to switch DNS over to it temporarily, but unbound seems to be working...for now.
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on June 24, 2021, 01:39:16 am
And now it is back and even worse than before. I haven't touched anything setting wise...
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on June 24, 2021, 02:18:58 am
I just removed my custom config for NextDNS and changed things to Cloudflare DoT.

Going to test and see if it works or not. At least I can eliminate something from the list.
Title: Re: Unbound DNS Locking Up
Post by: opnfwb on June 24, 2021, 06:44:06 pm
Just to confirm, are you saying that the Unbound service is stopping/crashing? Do you see any errors in the log file?

I've been using a custom config forwarding DoT to Quad9 for years as soon as it was supported by Unbound back in 2018. This has been very stable and the Unbound service itself has never shown any issues. There may be some clues in the log if it's a DNS provider problem.
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on June 25, 2021, 12:39:38 am
Just to confirm, are you saying that the Unbound service is stopping/crashing? Do you see any errors in the log file?

I've been using a custom config forwarding DoT to Quad9 for years as soon as it was supported by Unbound back in 2018. This has been very stable and the Unbound service itself has never shown any issues. There may be some clues in the log if it's a DNS provider problem.

As best as I can tell unbound is stopping. There is little if anything in the log to indicate such, but the symptoms are pointing to it (lookups fail, but can still ping ip addresses) and restarting the service an/or opnsense fixes it.

The problem has not occurred since I just changed to Cloudflare as of last night. Makes me wonder if mixing Unbound and NextDNS is the problem.

Title: Re: Unbound DNS Locking Up
Post by: chrisg11 on June 25, 2021, 02:17:20 am
I'm getting `status: REFUSED` DNS responses at times from Unbound according to dig lookups, with dig complaining about recursion not being available. Restarting Unbound "fixed" it at the moment but don't have confidence this will stay that way.

Code: [Select]
% dig example.com

; <<>> DiG 9.10.6 <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 55088
;; flags: qr rd ad; QUERY: 0, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; Query time: 67 msec
;; SERVER: 2601:18d:xxxx:xxxx:xxx:xxxx:xxxx:xxxx#53(2601:18d:xxxx:xxxx:xxx:xxxx:xxxx:xxxx)
;; WHEN: Thu Jun 24 20:10:03 EDT 2021
;; MSG SIZE  rcvd: 12

To be clear, not using DoT or an external resolver, I'm letting Unbound perform recursion itself. Also don't see anything obvious in the logs.
Title: Re: Unbound DNS Locking Up
Post by: opnfwb on June 25, 2021, 04:08:36 pm
As best as I can tell unbound is stopping. There is little if anything in the log to indicate such, but the symptoms are pointing to it (lookups fail, but can still ping ip addresses) and restarting the service an/or opnsense fixes it.
When you login to OPNsense, does Unbound have a red icon here instead of a green one (see attached screenshot)? Red would indicate the service stopped and/or crashed. If it's still green and DNS is not working, that indicates either a config issue or an issue somewhere else on the network (route issue, provider issue, etc.)

I'm getting `status: REFUSED` DNS responses at times from Unbound according to dig lookups, with dig complaining about recursion not being available. Restarting Unbound "fixed" it at the moment but don't have confidence this will stay that way.

Code: [Select]
% dig example.com

; <<>> DiG 9.10.6 <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 55088
;; flags: qr rd ad; QUERY: 0, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; Query time: 67 msec
;; SERVER: 2601:18d:xxxx:xxxx:xxx:xxxx:xxxx:xxxx#53(2601:18d:xxxx:xxxx:xxx:xxxx:xxxx:xxxx)
;; WHEN: Thu Jun 24 20:10:03 EDT 2021
;; MSG SIZE  rcvd: 12

To be clear, not using DoT or an external resolver, I'm letting Unbound perform recursion itself. Also don't see anything obvious in the logs.
This seems to be a different issue, potentially related to this maybe? https://github.com/NLnetLabs/unbound/issues/360

Also the IP listed in the dig command is IPv6. Is your network fully dual stack or would it have an issue resolving a DNS request to an IPv6 destination?
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on June 25, 2021, 08:03:33 pm
I'm not at home now to test, but the Unbound service always appears to be running, but no lookups will work.

My best guess at this point is that something about the combination of NextDNS and Unbound is causing lookups to just stall out permanently. Its not that the service is crashing, which would show up in the logs, its just stalling out until I end up rebooting it. But that is just my guess.

It looks like NextDNS has issues with unbound, but its a little above my head about what they are referring to in the GitHub post.

https://github.com/NLnetLabs/unbound/issues/132
Title: Re: Unbound DNS Locking Up
Post by: Msan on June 26, 2021, 03:35:11 am
i also have the same issue.. unbound is green and running but dns is not working.
i am also using nextdns.. seems to happen every 2 days or so..
i am going to switch to 1.1.1.1 and 8.8.8.8 to see if that makes any difference..
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on June 26, 2021, 07:45:18 am
i also have the same issue.. unbound is green and running but dns is not working.
i am also using nextdns.. seems to happen every 2 days or so..
i am going to switch to 1.1.1.1 and 8.8.8.8 to see if that makes any difference..

I have been running two days now on Cloudflare with the exact same settings, without once freezing up. So that would lead me to believe that my assumption was true about NextDNS and Unbound having problems when used together.

It does look like the NextDNS client on GitHub will work on OPNSense; much the same as it will on PfSense, seeing as they are both on FreeBSD.

I will likely end up going that route and disabling Unbound, once I get the chance to test it out.

https://github.com/nextdns/nextdns/wiki
Title: Re: Unbound DNS Locking Up
Post by: Mr.Goodcat on June 26, 2021, 02:24:21 pm
Here the issue occured as well. Unbound is configured to forward requests to DNSCrypt-Proxy. Somehow DNS just stops, but is shown as up an running. After a reboot it functions normally, with the exception, that DNSCrypt-Proxy doesn't show anything in its logs ??? It doesn't even report any basic information, such as connected servers or incomming DNS queries.
Title: Re: Unbound DNS Locking Up
Post by: Demus4202 on July 01, 2021, 04:40:21 pm
It has been about a week on Cloudflare DNS and not a single issue as everyone has described.

Must be an incompatibility somewhere as I and other posters have suggested previously.

Ill likely end up migrating to the NetCloud client when I have time here soon.
Title: Re: Unbound DNS Locking Up
Post by: dinguz on July 01, 2021, 04:51:52 pm
Some people have reported problems with unbound locking up, which boiled down to the system running out of resources. Have you checked cpu load, and memory and disk usage? Unbound on OPNsense seems particularly affected by /var running out of space.