Messages

I woke up to email this morning from Maxmind warning me that I had exceeded my daily download quota for GeoLite2.   I figured it was Opnsense. (Was running a 24.1 release until a few hours ago.)

Firewall Aliases GeoIP settings reports:

Code Select Expand

Last updated 2025-02-04T15:01:24
 Total number of ranges 1000178

Syslog has many entries of the form:

Code Select Expand

2025-02-06T08:25:12 Notice firewall geoip updated (files: 0 lines: 0)

Maxmind's site reports that I was downloading the GeoLite2-Country-CSV_20250204.zip file once or twice a second, whereas until this morning, it would download once per day.

Code Select Expand

GeoLite2-Country-CSV_20250204.zip 2/6/25 8:24
GeoLite2-Country-CSV_20250204.zip 2/6/25 8:24
GeoLite2-Country-CSV_20250204.zip 2/6/25 8:24
GeoLite2-Country-CSV_20250204.zip 2/6/25 8:24
GeoLite2-Country-CSV_20250204.zip 2/6/25 8:24
GeoLite2-Country-CSV_20250204.zip 2/5/25 8:23
GeoLite2-Country-CSV_20250131.zip 2/4/25 8:22
GeoLite2-Country-CSV_20250131.zip 2/3/25 8:21
GeoLite2-Country-CSV_20250131.zip 2/2/25 8:20
GeoLite2-Country-CSV_20250131.zip 2/1/25 8:19
GeoLite2-Country-CSV_20250128.zip 1/31/25 8:18

That nothwithstanding, the files in /usr/local/share/GeoIP/alias are still dated yesterday, so the update from Maxmind isn't completing.

I've disabled the Maxmind URL for now, and upgraded to 24.7.12.  Anyone else encounter this?  I took at look at this file and I don't see any way that this code itself could loop, but I don't know the larger context in which it could be called:

Code Select Expand

https://github.com/opnsense/core/blob/stable/24.1/src/opnsense/scripts/filter/lib/alias/geoip.py

My logs don't show any error messages that the code might emit, but they don't go back far enough either, due to log rotation. I would try fetching the zip file manually and analyzing it for corruption, but Maxmind has blocked me for a day...

I didn't see any system errors in the log. 

For now, I've installed a new SSD, imaged a fresh 24.1 deployment, restored all but the Unbound DNS configuration settings, and then manually recreated the Unbound DNS settings I want.  This works.  But for what it's worth, I tried restoring the last full configuration backup that I took before I shut down the broken system into a Live CD session of 24.1. This produced exactly the same behavior as above.  I'm mystified, but happy to have a working DNS server in my network again.

I have a ProtectLi Intel Core i5 machine with 16 GiB of memory.  UnboundDNS spontaneously stopped responding to requests today.  I did not and had not in some weeks altered any settings (DNS or Firewall or Interface) on it.  I tried swapping dnsmasq for unbound, but get the same non responsiveness.

DNS requests using host/dig/nslookup time out, whether from on the opnsense machine itself or from a LAN host.  From a macOS LAN client, host -T fails immediately:

$ host -T btc.i2kconnect.com 192.168.0.1
;; communications error to 192.168.0.1#53: network down

But host -T from the opnsense machine times out.

unbound-control can't talk to it either, running from an opnsense-shell on the router and trying to access its control port 953 on its local IP address or on its loopback address 127.0.0.1.  I was running opnsense 23.7 when this happened, and upgraded to 24.1 in desperation but this made no difference.   I'm not seeing any packets dropped by the firewall. 

sockstat indicates that unbound is listening to port 53 for both TCP and UDP

root@btc-firewall:/var/log # sockstat -l -4 -p 53
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
unbound  unbound    25105 7  udp4   *:53                  *:*
unbound  unbound    25105 8  tcp4   *:53                  *:*
unbound  unbound    25105 11 udp4   *:53                  *:*
unbound  unbound    25105 12 tcp4   *:53                  *:*
unbound  unbound    25105 15 udp4   *:53                  *:*
unbound  unbound    25105 16 tcp4   *:53                  *:*
unbound  unbound    25105 19 udp4   *:53                  *:*
unbound  unbound    25105 20 tcp4   *:53                  *:*
root@btc-firewall:/var/log #

Once unbound starts up, there is no traffic in the unbound log either, as shown below.  Other than unbound/dnsmasq, the machine is routing as expected. 

Since the problem affects both dnsmasq and unbound, I suspect the problem is not the DNS services themselves, but I can't imagine what could be blocking the request traffic.  Any suggestions for how to proceed would be greatly welcomed.

2024-06-01T23:21:31   20   Notice   unbound   31787   Backgrounding unbound logging backend.   
2024-06-01T23:21:31   3   Informational   unbound   25105   [25105:0] info: dnsbl_module: updating blocklist.   
2024-06-01T23:21:30   20   Notice   unbound   29087   daemonize unbound dhcpd watcher.   
2024-06-01T23:21:30   3   Notice   unbound   25105   [25105:0] notice: init module 0: python   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: module config: "python iterator"   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 198.41.0.4 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:503:ba3e::2:30 port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 170.247.170.2 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2801:1b8:10::b port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 192.33.4.12 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:2::c port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 199.7.91.13 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:2d::d port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 192.203.230.10 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:a8::e port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 192.5.5.241 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:2f::f port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 192.112.36.4 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:12::d0d port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 198.97.190.53 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:1::53 port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 192.36.148.17 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:7fe::53 port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 192.58.128.30 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:503:c27::2:30 port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 193.0.14.129 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:7fd::1 port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 199.7.83.42 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:500:9f::42 port 53 (len 28)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 202.12.27.33 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip6 2001:dc3::35 port 53 (len 28)   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: A.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: B.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: C.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: D.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: E.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: F.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: G.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: H.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: I.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: J.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: K.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: L.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: M.ROOT-SERVERS.NET. * A AAAA   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: DelegationPoint<.>: 13 names (0 missing), 26 addrs (0 result, 26 avail) parentNS   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: Reading root hints from /root.hints   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 208.67.220.220 port 53 (len 16)   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ip4 208.67.222.222 port 53 (len 16)   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: DelegationPoint<.>: 0 names (0 missing), 2 addrs (0 result, 2 avail) parentNS   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: Forward zone server list:   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: 1.0.0.127.in-addr.arpa. PTR localhost   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: localhost A 127.0.0.1   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa. PTR localhost   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: localhost AAAA ::1   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: 1.0.168.192.in-addr.arpa. PTR btc-firewall.i2kconnect.com   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: btc-firewall.i2kconnect.com A 192.168.0.1   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: 200.0.168.192.in-addr.arpa. PTR btc.i2kconnect.com   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: btc.i2kconnect.com IN A 192.168.0.200   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: 201.0.168.192.in-addr.arpa. PTR btc-master.i2kconnect.com   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: ignoring duplicate RR: btc-master.i2kconnect.com IN A 192.168.0.201   
2024-06-01T23:21:30   3   Informational   unbound   25105   [25105:0] info: implicit transparent local-zone . TYPE0 IN   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: drop user privileges, run as unbound   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: chroot to /var/unbound   
2024-06-01T23:21:30   3   Debug   unbound   25105   [25105:0] debug: chdir to /var/unbound