Very slow DNS for updates, plugin listing, remote calls from the OPNSense box

[harg]

Router info:

Code: [Select]

*** myrouter.localdomain: OPNsense 23.1.9 ***

 LAN (igc1)      -> v4: 192.168.1.1/24
                    v6/t6: 2a0d:3344:12b:c900:62be:b4ff:****:****/64
 Starlink (igc0) -> v4/DHCP4: 100.82.***.***/10
                    v6/DHCP6: fe80::62be:b4ff:****:****%igc0/64


My setup is pretty simple; a single WAN interface (Starlink) and a single LAN interface. I'm also running Adguard home as the DNS for the LAN DHCP server.

Clients on the LAN can access the internet fine and DNS seems fast. ipv6 seems to also work.

However I noticed that when visiting System->Firmware->Status it would be very slow to show any information (up to ~5mins) or would fail to show anything at all. The same applies to the plugins & packages tabs.

It seemed like a DNS issue so I ssh'd into the OPNSense box and tried pinging some hosts. I noticed there was a significant delay after running `ping` and any output appearing:

Code: [Select]

root@myrouter:~ # ping google.com
# delay of ~20-30 seconds
PING google.com (142.250.200.14): 56 data bytes
64 bytes from 142.250.200.14: icmp_seq=0 ttl=119 time=43.432 ms
64 bytes from 142.250.200.14: icmp_seq=1 ttl=119 time=58.192 ms

If I ping the IP address directly the output is instant:

Code: [Select]

root@myrouter:~ # ping 142.250.200.14
PING 142.250.200.14 (142.250.200.14): 56 data bytes
64 bytes from 142.250.200.14: icmp_seq=0 ttl=119 time=48.951 ms

It seems like OPNSense itself is having trouble resolving DNS. It seems to work eventually but making anything that involves resolving a domain very slow. As mentioned this does not seem to affect clients on the LAN where Adguard Home is being using for DNS; just calls from the OPNSense box itself.

Any advice on how to diagnose and fix the issue?

[CJ]

How do you have the OPNSense DNS configured?  Did you change it to use AdGuard or is it still using your ISP DNS?

You can use the DNS Lookup under Interfaces->Diagnostics to test queries.  No need to ssh in.  Same with ping.

[harg]

I haven't configured the OPNSense DNS so I guess it's using ISP DNS. I've left the "DNS Servers" section blank under System->Settings->General.

[CJ]

I haven't configured the OPNSense DNS so I guess it's using ISP DNS. I've left the "DNS Servers" section blank under System->Settings->General.

What about the two DNS checkboxes under that section?  Do you have any DNS firewall rules in place?

What results do you get when you test using the Diagnostics pages?

[harg]

Re-igniting this. Sorry, I was away for a few weeks.

This is what the settings page looks like re DNS:


pic host site

I haven't set any extra firewall rules; I'm using the default ones set by OPNSense. Here're my floating rules:


pic host site

What kind of test could I run from the diagnositics pages?

[CJ]

Go to Interfaces -> Diagnostics -> DNS Lookup and put in google.com and post the result.

What do you have under Query Forwarding and DNS over TLS under Unbound?

Lastly, what do you have on your DHCP settings for DNS?

[harg]

Thanks for the reply. DNS lookup worked with the following result:



Even though it says ~28ms query time, it took about 20-30 seconds from clicking the button to the results appearing.

Re Unbound, it's empty for DNS over TLS and Query forwarding; but I'm not using Unbound afaik ("enable" is unchecked in the Unbound settings).

For DNS under DHCP, on the LAN interface I have the router IP set as the DNS server, as I'm running Adguard Home as my DNS server on the same box. Devices on the LAN don't seem to have any trouble with DNS lookups or slowness - it's just with the OPNSense box itself, e.g. if I ssh into it and perform a ping to some external domain.

This is the Adguard Home settings in the OPNSense admin:

[CJ]

Thanks for the reply. DNS lookup worked with the following result:



Even though it says ~28ms query time, it took about 20-30 seconds from clicking the button to the results appearing.

That's odd.  Under Interfaces -> Overview -> WAN, what do you have for DNS Servers?  With an empty settings list and allow DNS to be overridden, I would expect to see your ISP servers, not Google DNS.

Are you running dual stack?  How is your IPv6 configured?

What happens if you do a query but put 127.0.0.1 in the Server field?

Re Unbound, it's empty for DNS over TLS and Query forwarding; but I'm not using Unbound afaik ("enable" is unchecked in the Unbound settings).

Right.  I forgot you're running AdGuard.  There's a bunch of these DNS threads and it's hard to keep the details straight.

For DNS under DHCP, on the LAN interface I have the router IP set as the DNS server, as I'm running Adguard Home as my DNS server on the same box. Devices on the LAN don't seem to have any trouble with DNS lookups or slowness - it's just with the OPNSense box itself, e.g. if I ssh into it and perform a ping to some external domain.

Understood.  That's because by default OPNSense will use your ISP DNS servers while handing out Unbound to your clients which uses the root DNS servers.  Which would mean that there's something wrong with your ISP DNS servers.  But your setup is a bit more unusual.

This is the Adguard Home settings in the OPNSense admin:

I'm not familiar with Adguard but I would have expected to see it used in the DNS Lookup, showing as Server 127.0.0.1.  It looks like that only works for Dnsmasq and Unbound, though.

What do you have set as your upstream DNS providers in Adguard?

[harg]

Are you running dual stack?  How is your IPv6 configured?

Yes, I am running dual stack. WAN is set to DHCPv6 for ipv6. ISP is Starlink. Here is the ipv6 settings for WAN, which I got from following a guide somewhere:

What happens if you do a query but put 127.0.0.1 in the Server field?

it times out and I get an error: "Error: error sending query: Could not send or receive, because of network error"

However, if I use 1.1.1.1 or 8.8.8.8 it works immediately

[CJ]

Are you running dual stack?  How is your IPv6 configured?

Yes, I am running dual stack. WAN is set to DHCPv6 for ipv6. ISP is Starlink. Here is the ipv6 settings for WAN, which I got from following a guide somewhere:

I don't think that should be causing this.  What about the rest of the things I asked?  Your Interface info and AdGuard settings?

What happens if you do a query but put 127.0.0.1 in the Server field?

it times out and I get an error: "Error: error sending query: Could not send or receive, because of network error"

However, if I use 1.1.1.1 or 8.8.8.8 it works immediately

That's to be expected.  Your problem is your local dns resolution and why I need all of the other info I asked for in the previous post.

[Maurice]

OPNsense queries localhost first, doesn't get a response, times out, then queries the DNS servers assigned by the ISP (afaik Starlink indeed uses Google's DNS servers). The delay is because of waiting for a response from localhost before switching to Google DNS.

I'm not familiar with the AdGuard settings. Maybe you configured it to not bind to localhost? If you want to use AdGuard for OPNsense itself, you'll have to change this. Otherwise, just enable "Do not use the local DNS service as a nameserver for this system". OPNsense will then go to Google DNS directly.

Cheers
Maurice

[harg]

Apologies, I was a away from this router for a bit.

I had a poke around the adguard home settings for a bit but I couldn't find anything that seemed to control whether it was binding to localhost or not.

I tried your other suggestion of checking the "Do not use the local DNS service as a nameserver for this system" box and it's solved the issue! So many thanks!

I'm not too bothered with not using adguard home for the router DNS; it's mostly to make web browsing more bearable.

Thanks again for the help. I think we can call this question answered.

Update:

Upon investigating further, I found that Adguard Home was indeed only listening on the LAN interface. It seems like it's not possible to change this via the web interface but I was able to make it listen on all interfaces by editing the config file in `/usr/local/AdGuardHome/AdGuardHome.yaml` and restarting the service.