So here is the thing I for the moment don't get:
I earlier today turned on Unbound DNS BlackList, and checked them all. This used to work on my old OPNsense firewall (hardware broke on that one, so I building a new one).
So when enabled, BlackLists, I get a delay of 10-15 seconds before internet (DNS resolution) wakes up when I wake up my PC on the LAN interface. If I disable BlackLists, it works direct after I wake up my LAN connected PC.
Anyone know why this is?
It's possible the computer is trying to check something dns related that is being blocked by the backlists. Have you checked to see if windows determines if you have internet access when the internet is not working?
Well I kind of got there also.
So I disables the WindowsSpy stuff (three lists) in the bottom, made no difference.
So then I disabled everything except the first one (AdAwayList) - this worked.
Will now work my way thru each and everyone of them to see when I hit the same issue again. I'll be back...
EDIT: Windows says I have internet access all the time - it is just DNS resolution that seems to fail...
EDIT (again): The current OPNsense hardware, since the previous one got h/w failure, sits inside another firewall that happens to run a few blacklists also...
Okay - a lot of testing later...
The delay could be related to some sort of volume of selected BlackLists....
Since I get longer and longer delay as more of the BlackLists get selected...
Why is Unbound restarted every time someone starts the PC connected to LAN interface is woken up?
The PC connected to that single interface, LAN, goes to sleep and then the traffic on that interface, LAN, stops. At that time nothing seems to happen in the log, however as soon as someone (most likly me) starts that only connected PC, Unbound restarts:
2020-12-07T06:37:00 unbound[19877] [19877:1] info: generate keytag query _ta-4f66. NULL IN
2020-12-07T06:37:00 unbound[19877] [19877:2] info: generate keytag query _ta-4f66. NULL IN
2020-12-07T06:37:00 unbound[19877] [19877:3] info: generate keytag query _ta-4f66. NULL IN
2020-12-07T06:37:00 unbound[19877] [19877:0] info: start of service (unbound 1.12.0).
2020-12-07T06:37:00 unbound[19877] [19877:0] notice: init module 1: iterator
2020-12-07T06:37:00 unbound[19877] [19877:0] notice: init module 0: validator
2020-12-07T06:36:19 unbound[19877] [19877:0] notice: Restart of unbound 1.12.0.
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.032768 0.065536 2
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.008192 0.016384 2
2020-12-07T06:36:19 unbound[19877] [19877:0] info: lower(secs) upper(secs) recursions
2020-12-07T06:36:19 unbound[19877] [19877:0] info: [25%]=0.012288 median[50%]=0.016384 [75%]=0.049152
2020-12-07T06:36:19 unbound[19877] [19877:0] info: histogram of recursion processing times
2020-12-07T06:36:19 unbound[19877] [19877:0] info: average recursion processing time 0.029205 sec
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 7: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 7: 25 queries, 21 answers from cache, 4 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.016384 0.032768 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: lower(secs) upper(secs) recursions
2020-12-07T06:36:19 unbound[19877] [19877:0] info: [25%]=0 median[50%]=0 [75%]=0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: histogram of recursion processing times
2020-12-07T06:36:19 unbound[19877] [19877:0] info: average recursion processing time 0.030775 sec
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 6: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 6: 13 queries, 12 answers from cache, 1 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 5: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 5: 2 queries, 2 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 4: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 4: 3 queries, 3 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.131072 0.262144 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.008192 0.016384 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: lower(secs) upper(secs) recursions
2020-12-07T06:36:19 unbound[19877] [19877:0] info: [25%]=0 median[50%]=0 [75%]=0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: histogram of recursion processing times
2020-12-07T06:36:19 unbound[19877] [19877:0] info: average recursion processing time 0.087900 sec
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 3: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 3: 7 queries, 5 answers from cache, 2 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.016384 0.032768 2
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.008192 0.016384 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.000000 0.000001 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: lower(secs) upper(secs) recursions
2020-12-07T06:36:19 unbound[19877] [19877:0] info: [25%]=1e-06 median[50%]=0.016384 [75%]=0.024576
2020-12-07T06:36:19 unbound[19877] [19877:0] info: histogram of recursion processing times
2020-12-07T06:36:19 unbound[19877] [19877:0] info: average recursion processing time 0.010952 sec
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 2: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 2: 12 queries, 8 answers from cache, 4 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.131072 0.262144 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.065536 0.131072 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.032768 0.065536 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.016384 0.032768 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: lower(secs) upper(secs) recursions
2020-12-07T06:36:19 unbound[19877] [19877:0] info: [25%]=0.032768 median[50%]=0.065536 [75%]=0.131072
2020-12-07T06:36:19 unbound[19877] [19877:0] info: histogram of recursion processing times
2020-12-07T06:36:19 unbound[19877] [19877:0] info: average recursion processing time 0.082588 sec
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 1: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 1: 9 queries, 5 answers from cache, 4 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.262144 0.524288 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.016384 0.032768 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: 0.000000 0.000001 1
2020-12-07T06:36:19 unbound[19877] [19877:0] info: lower(secs) upper(secs) recursions
2020-12-07T06:36:19 unbound[19877] [19877:0] info: [25%]=0 median[50%]=0 [75%]=0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: histogram of recursion processing times
2020-12-07T06:36:19 unbound[19877] [19877:0] info: average recursion processing time 0.174345 sec
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
2020-12-07T06:36:19 unbound[19877] [19877:0] info: server stats for thread 0: 10 queries, 7 answers from cache, 3 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2020-12-07T06:36:19 unbound[19877] [19877:0] info: service stopped (unbound 1.12.0).
Now since Unbound decides to restart, for what evere reason, the DNS resolution will not be available since well it may have started, but it for sure takes a while to process the BlackLists that are active - the more BlackList - the longer the process time (logical, although with 8 cores i7 as CPU well...).
So why is Unbound restarted then? It's not like I added an interface - I just resumed operation on the LAN interface...?
is DHCP registration enabled on unbound?
Quoteyes
so. unbound needs to re-read config with new entry after dhcp-client address registration.
and the larger the file with dnsbl records, the longer it takes to start
Quote from: Fright on December 07, 2020, 07:39:53 AM
Quoteyes
so. unbound needs to re-reade config with new entry after dhcp-client address registration.
and the larger the file dnsbl records, the longer it takes to start
So what you are saying is that I need to turn off, disable, DHCP registration within Unbound to stop this? I'll give it a try.
Quotedisable, DHCP registration within Unbound to stop this?
yes. or use less dnsbl-s
if you don't have many hosts to register, you can try to solve this with Overrides and dhcp reservations
Quote from: Fright on December 07, 2020, 07:45:06 AM
Quotedisable, DHCP registration within Unbound to stop this?
yes. or use less dnsbl-s
if you don't have many hosts to register, you can try to solve this with Overrides and dhcp reservations
No.
Made no difference at all, still get Unbound restart.
The thing is here, since Unbound handles all DNS requests, all ports so to speak goes down when a interface that has had no traffic (PC/Server is power off or just sleeping) triggers a restart of Unbound. Even without any blacklists enabled this still happens. And you do get downtime. With blacklists it is just much worse.
QuoteThe thing is here, since Unbound handles all DNS requests, all ports so to speak goes down when a interface that has had no traffic (PC/Server is power off or just sleeping) triggers a restart of Unbound. Even without any blacklists enabled this still happens. And you do get downtime. With blacklists it is just much worse.
did not understand the idea
QuoteEven without any blacklists enabled this still happens
if DHCP Registration enabled this will happen when the client receives the lease. it will just be faster if the dnsbl list is small or disabled
Under Unbound -> General settings:
DHCP Registration is disabled
DHCP Static Mappings is disabled
Network Interfaces is (for the moment) LAN and WORK
Outgoing Network Interfaces (for the moment) WAN_FTTH and WAN_LTE (failover)
And yes Unbound is enabled.
No matter what status BlackLists are in (enabled or disabled), Unbound will restart no matter what when for example LAN interface wakes up.
Since i have 8 ports available (only 4 used so far, I am still building my new OPNsense), when I will enable the last 4 I will also connect a lot more stuff that will need DNS resolution. Some of them might go into powersave and when it wakes up, I guess Unbound will be triggered to reboot. Just as when the now PC connected to LAN port wakes up.
This will interupt DNS service since Unbound is unavailable.
On my other firewall, which happens to use Unbound also - this does not happen, and well that Unbound installation runs a lot of block lists also. No interruption at all.
Also my previous OPNsense box, I don't remember this to happening so it feels like something new (to me at least).
Quotesome of them might go into powersave and when it wakes up, I guess Unbound will be triggered to reboot.
sorry. didn't know about powersave.
yes. it triggers restart afaik
Okay - So no matter how I try to solve this, if a host / server / what ever goes into powersave / turned off - and then power on again, the end result is that Unbound will restart.
I think this is the third thing I might not have solved this way.
sorry, let's clarify..
OPN interface UP\DOWN or some clients interfaces up\down?
clients interfaces states should not triggeer undound restart. unless they make changes to the unbound zones
Client ethernet port, in this case my PC connected to the LAN interface with cable 1:1, will be turned off during client powersave. Nothing changes on that client, or in OPNsense. It is just a powersave off/on cycle.
Just wanted to let you know that I had an old, invalid entry under System - Gateways - Single. Since I removed it, the problem is gone.
Thanks for the tip. The only two entries I have are WAN_FTTH and WAN_LTE (failover) - they have to exist.
QuotePC connected to the LAN interface with cable 1:1, will be turned off during client powersave
Oh. it explains ..
can you try to bind unbound only to constantly raised interfaces and use nat?
upd. not works with "always-UP" interface binding on test VM, sorry
looks like OPN configure hosts record on unbound and kills unbound process if any interface change state to up
(it is logical that the service binding is not taken into account. but the existence of a record or a static address is also not checked)
Okay, so why does Unbound depend som much on interface status, that it needs to restart all the time?
I guess if I would set up pi-hole which - if my memory is correct - also uses Unbound, don't seem to need to restart?
Quoteso why does Unbound depend som much on interface status, that it needs to restart all the time?
if I understand correctly, OPN scripts try to keep the content of the zones up to date. but unbound only loads them on start - that's the problem. almost imperceptible if the config is small and PITA if it is large.
(probably it would be possible to compare the configurations and apply them (by restarting unbound) only if they really changed. this would reduce the number of restarts)
Quoteguess if I would set up pi-hole which - if my memory is correct - also uses Unbound, don't seem to need to restart?
its on dnsmasq if I remember it right.can't say much about pi-hole
Quote from: Fright on December 07, 2020, 07:53:09 PM
Quoteguess if I would set up pi-hole which - if my memory is correct - also uses Unbound, don't seem to need to restart?
its on dnsmasq if I remember it right.can't say much about pi-hole
You are correct, of course. I have read a number of articles that use pi-hole as filter and then uses a local Unbound installation. That is why I remember it incorrect.
I do have an Ubuntu server that could be used for a pi-hole installation and do all the filtering, then use OPNsense with Unbound DNS. But it feels so wrong, like going over the river for the water.....
QuoteBut it feels so wrong, like going over the river for the water.....
so you may can try to use switch(es), less DNSBLs and update dnsbls at night? )
DNSBL are only updated once a week, at 03.00. So no that will not change anything. The only way to work around this is by removing all DNSBL lists, and simply never use OPNsense for DNS resolution.
Just to verify, I disabled Unbound, installed BIND (I'm not a fan of BIND), set it up, and yes it works without loosing DNS while wakeup of interface. However there is no support for DoT on forward, and no custom blocklists, and maybe more...
Quoteno support for DoT on forward, and no custom blocklists
I think this is fixable: a couple of changes and hooks in the template and dnsbl script. plus nginx or stunnel for tls proxy.
but I cannot understand just one thing: if the problems starts with powersave clients, why not use a switch so that the OPN interface does not goes down/up?
Quote from: Fright on December 10, 2020, 07:01:28 AMbut I cannot understand just one thing: if the problems starts with powersave clients, why not use a switch so that the OPN interface does not goes down/up?
Since 2 (two) out of 8 (eight) ports will have direct connection: LAN which has a direct hard line, if I might use that term, to one particular PC and Printer/Scanner to another one. The rest are servers (online 24x7) and switches (runs also, of course, 24x7). Now I have not tested the printer/scanner if it goes down into powersave in that sense - however I have tested LAN, and I have no plans to get a switch for that one since that is the one controlling (so to speak) OPNsense.
And again, switches are just bandaids. I have been reading up on this issue (which I like to call it) and it turns out that when an DHCP address is changed there is a call to Unbound using HUP method - restart. And then we have the Zone change, which might occur anytime (?), which also seems to use HUP method - restart. Both can be handled with a Unbound reload local zone instead, but that has never materialized even though both pfsense and OPNsense has the same issue. I am not saying it is an easy fix (I have just started to try to look into the code, and so far not found what I am looking for - so for me it is impossible for the moment to even see if I could solve this), I think it is bigger than that. However I wanted to test if it could be solved with a change to BIND (which at least I have worked with, but like 10 years ago, at least that is how it feels...). For the moment I am thinking of changing to DNScypt-Proxy just to test that plugin out. However I still think it will not allow custom black lists. Although it does use DoH (instead of DoT) so it is a better solution on paper...
Best would be if, somehow, Unbound could be improved.
QuoteBest would be if, somehowe, Unbound could be improved.
true!
Do we (?) know if a server restart/reboot (Windows/Linux) will trigger a reload also in Unbound, if it is direct connected to OPNsense (like my LAN PC is connected)?
I can't know for sure, but logically - it should, because the physical link will be down / up
Quote from: lar.hed on December 10, 2020, 01:38:56 PM
Do we (?) know if a server restart/reboot (Windows/Linux) will trigger a reload also in Unbound, if it is direct connected to OPNsense (like my LAN PC is connected)?
Okay, I just had to test this: restart of server = restart of Unbound, and if one has a bunch of BlockLists, well OPNsense in my case looses name resolution for about 40 seconds = downtime. Sorry.
This in my case also gives that 4 (four and not 2) out of 8 ports are in risk of triggering a Unbound restart = downtime of about 40 seconds. Adding a managed switch, which of course will solve the restart Unbound issue, will be way more complicated with VLANs and all. It is sooo much simpler with just raw ports direct into OPNsense (in my mind, anyone is free to have other opinions).
I'll test DNS-Proxy now...
Quote from: lar.hed on December 10, 2020, 04:27:43 PMI'll test DNS-Proxy now...
Yes this works kind of, I get a downtime of about 8 seconds before everything works - however I do get an answer without reload of a web page for example - I am not loosing any info as far as I can tell. So Yes it kind of works - it just needs about 8 seconds to local zone to be reloaded - but that is it.
I have to say, DNSCrypt-Proxy looks good so far. Does not support local blocklists (I have two that I need for Sweden only so to speak, so being able to add them is well something I would like, anyone knows if they could be added via ssh/shell?), but the rest is a-okay. And I have done some more testing, and there does not seem to be any 8 seconds wait like I had in the first few tries - maybe I was a bit to eager...
Quoteanyone knows if they could be added via ssh/shell?
I think you can. via custom configd action
you can copy the update script with hardcoded dnsbls (/usr/local/opnsense/scripts/OPNsense/Dnscryptproxy/dnsbl.sh), add your lists to it (don't forget to clear it. it should only contain fqdns), add a custom action to configd (copy of "Download DNSCrypt-Proxy DNSBLs and restart" but with your own script and Description) and cron this job instead of a "Download DNSCrypt-Proxy DNSBLs and restart"
(imo its better to ask the devs (
@mimugmail I think) ) to add some kind of hook to the dnsbl.sh for custom dnsbls (source "$custom_dnsbls" ? ))
QuoteAdding a managed switch, which of course will solve the restart Unbound issue, will be way more complicated with VLANs and all
I thought that "protected ports" would be enough for it
Quote from: Fright on December 11, 2020, 07:20:51 AM(imo its better to ask the devs (@mimugmail I think) ) to add some kind of hook to the dnsbl.sh for custom dnsbls (source "$custom_dnsbls" ? ))
Although that would be great, I rather see that any time spent on this issue is spent on Unbound.
Quote from: Fright on December 11, 2020, 07:20:51 AM
Quoteanyone knows if they could be added via ssh/shell?
I think you can. via custom configd action
you can copy the update script with hardcoded dnsbls (/usr/local/opnsense/scripts/OPNsense/Dnscryptproxy/dnsbl.sh), add your lists to it (don't forget to clear it. it should only contain fqdns), add a custom action to configd (copy of "Download DNSCrypt-Proxy DNSBLs and restart" but with your own script and Description) and cron this job instead of a "Download DNSCrypt-Proxy DNSBLs and restart"
Yep - that seems to work just perfect. For now, or until Unbound is just that tiny bit better with reload instead of HUP, this will do just super. And my WAN fail-over also works like a charm - so happy for the moment!
(until I get to next challenge where I most likely will have to figure something else out - gray hair or what it now was according to @mimugmail )
can you test unbound.inc changes for if down/up unbound HUP issue workaround? (checks if a file has changed before reloading). works on test VM but who knows..
https://github.com/kulikov-a/core/commit/d799447ae87bd722a301c83822cf7671c088ee9e
Quote from: Fright on December 11, 2020, 10:07:07 PM
can you test unbound.inc changes for if down/up unbound HUP issue workaround? (checks if a file has changed before reloading). works on test VM but who knows..
https://github.com/kulikov-a/core/commit/d799447ae87bd722a301c83822cf7671c088ee9e
Works 100% perfect!
Good!
I also noticed that the unbound does not actually restarts on cron job "Download Unbound DNSBLs and restart".
imo the command in actions_unbound.conf should be changed to
/usr/local/opnsense/scripts/unbound/download_blacklists.py && configctl unbound reload
can also change download_blacklists.py a little
before:
blacklist_items.add(entry)
after:
blacklist_items.add(entry.lower())
(in debug mode there is many messages about a duplicate dnsbl when local zone loads. but it does not affect the load time i think)
it might be worth making a FR for unbound.inc\actions_unbound.conf changes
Quote from: Fright on December 12, 2020, 08:07:03 AM
I also noticed that the unbound does not actually restarts on cron job "Download Unbound DNSBLs and restart".
imo the command in actions_unbound.conf should be changed to
/usr/local/opnsense/scripts/unbound/download_blacklists.py && configctl unbound reload
Not sure I follow on this, since the code I have (not changed in any way) is:
[dnsbl]
command:/usr/local/opnsense/scripts/unbound/download_blacklists.py && /usr/local/sbin/pluginctl unbound restart
parameters:
type:script
message:fetching and applying DNSBLs
description: Download Unbound DNSBLs and restart
I'll have a look on the other stuff now...
Quote from: Fright on December 12, 2020, 08:07:03 AM
it might be worth making a FR for unbound.inc\actions_unbound.conf changes
Ohhh this is where my lack of (Git-) knowledge will be a bit embarrassing I guess, I have no idea how to do that :(
QuoteNot sure I follow on this
in your string its
pluginctl unbound restartdo nothing on my test VM. afaik it should be
pluginctl -s unbound restart or
configctl unbound reloadreload is far more faster than restart. on test VM with full DNSBLs list it takes 40sec to reaload and 1m24sec to restart
(need to talk with devs. imho reload is enough after dnsbl update)
QuoteI have no idea how to do that
ok. i'll try )
Aah - now I do follow! Thanks! I have no idea (yet) if the restart as it is written works or not, since well I have simply put not tested it (yet - but now I have to figure out a test).
I tried to reproduce this issue. Indeed, the whole unbound process is restartet in case a network links gets down and comes up again. This should not occur. But, now I know why sometimes name resolution stalls.
Do we have a related ticket on github regarding the FR?
FR added
https://github.com/opnsense/core/issues/4518
Interim results: "restart after dnsbl update" issue solved by @AdSchellevis before I noticed it ;D. Devs are aware of the problems with the unbound restart performance and are thinking about a solution. Suggestion to check host_entries file before restart (and skip restart if nothing changed) not accepted
What ever solution there might be in the end must be considered an improvement 8)
Good work there @Fright !
Btw just for facts: With my OPNsense h/w with a Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (8 cores), I restart (!) Unbound with all blocklists plus my own two in under 40 seconds. I have no idea how long a reload would take, but an estimate might be about 20 seconds?
easy to compare by checking unbound logs)
configctl unbound reloadfor reload
and
pluginctl -s unbound restarfor restart
Quotebut an estimate might be about 20 seconds?
I wouldn't be surprised if it fits in 15
Quote from: Fright on December 12, 2020, 04:55:05 PM
easy to compare by checking unbound logs:
configctl unbound reload
Well...reload according to the log, is 40 seconds also - now in the log it says restart.
Quotenotice: Restart of unbound 1.12.0.
I just noticed that Unbound gets restarted when one saves System -> Administration -> Settings (I was altering SUDO, to disallow since I do not like to have that one opened...). That was kind of a big surprise...!
QuoteWell...reload according to the log, is 40 seconds also
thanks for the info!
looked deeper ..a bit embarrassing - should have found out before creating a FR.
the difference in time between restarting and reloading (~50sec) is just in one line - the launch (and fail) of the unbound-anchor utility in unbound_start.sh. DNSSEC is not allowed, by the way.
Need a little more time to find out what is happening (it looks like something with DNS traffic during the launch of the unbound-anchor: I don't see any replies on dns OPT-requests).
with successful launch of the unbound-anchor, the time difference between restart and reload should be imperceptible i think
(at the same time, I think that there is no need to launch unbound-anchor if DNSSEC is not enabled)
Oh. old cisco pix two hops in front of test VM blocked dns packets larger than 512 bytes..
now it takes 2 sec to start unbound-anchor and 37 to load unbound. so (!with unbound-anchor successful launch!) there is no imperceptible difference in time between restart and reload. need to think whats next
I just reversed back to DNSCrypt-Proxy since well no downtime what-so-ever for regular stuff like link up/down, changes in System and so on - all that Unbound for some reason needs a restart = 40 seconds of downtime.
I am wondering if the unbound will restart in 1-2 sec (without reading the DNSBLs) and the DNSBLs will load into a running unbound, will this help? I want to test and discuss with the devs, but only if it makes sense
upd:
quickly tested this approach
result: no downtime at all at restart. the server serves requests while loading the full 102MB dnsbl list
of course, while the list is loading, accidentally resolving a name from the dnsbl-s is possible
QuoteI just noticed that Unbound gets restarted when one saves System -> Administration -> Settings
confirm. reloads after that. i think it make sense. page contains dns-relative parameters
Quote from: Fright on December 13, 2020, 04:31:26 PM
QuoteI just noticed that Unbound gets restarted when one saves System -> Administration -> Settings
confirm. reloads after that. i think it make sense. page contains dns-relative parameters
Hmmm well that does sound legit reason then, but the thing is I did not change any of that. I just altered the SUDO property. If I do that with DNSCrypt-Proxy this does not seem to happen - and the memory consumption is way less (I have 16GB RAM, and I can upgrade to 32GB if needed so I have no memory problem). Maybe DNSCrypt works in a different way?
Quotebut the thing is I did not change any of that
it seems to me that it is simply easier to respond to the applying of the entire form than to control and compare values from form fields and values in the system config.
QuoteMaybe DNSCrypt works in a different way?
developers can say more, but in my opinion the DNSCrypt is not so integrated into the whole system as an unbound (updating hosts records, registering dhcp client records etc.)
so what about loading dnsbls after unbound start (no downtime but accidentally resolving a name from the dnsbl-s is possible)? it might be interesting? for me this is more correct behavior than long service loading
Let me be cristal clear: Unbound without all restarts, or restarts with zero (or close to zero) DNS downtime is most likely the best solution, no question about this. However, current solution with all restarts needs some sort of work, at least when combined with DNS BlockLists. Removing BlockLists are not the solution, I simply want it all...
DNSCrypt-proxy is not the answeer either, since I need to hack a few things and that is not allowed in my book (and it should not be allowed in anys book, playaround sure not for production though). And still I have not figured out how to prevent Firefox from using DoT or DoH (which may or may not be working in Unbound either). It is fast though, no DNS downtime to talk about, and well no integration so one need to know and understand what needs to be configured.
BIND seems to work, but I consider it pain to configure correct (I also seem to have forgotten what I used to know). I like to stay away from BIND.
I think I need to add a few bits of information:
a) Hacking DNSCrypt-Proxy just showed why one should NEVER hack: I managed to loose all blocklists due to me thinking to much and trying to be overthink this. Solved of course, but never hack.
b) DNSCrypt for sure seems to start much faster - or let me phrase this like: DNSCrypt responds to DNS querys direct, however it does NOT seem to response to blocklists direct - that will take a few moments. So downtime is lower (of course) but blockslists are not loaded at restart - it takes about the same time as Unbound.
So Fright, your idea about starting and loading in two different processes is more or less what seems to happen in DNSCrypt-proxy. And yes that is more than okay - I need DNS resolution direct of course, blocking is important but DNS resolution is way more important. If one could get that into Unbound.... We would get a winner.
got it. I considering unbound as a forwarder for internal servers and also takes the speed and stability of servicing requests as a priority over dnsbls preparedness at start.
discussing the transition to dnsbl load with unbound-control in the same discussion at github