Unbound stopps suddenly

[Kinerg]

I have 4 GB of RAM in the system. The system is running at 30% capacity.
When unbound crashing it occurs after the DNS blocklist is updated,
which is done via cron once a day.
However, unbound does not crash every day after the cron job runs.

4 GB is not enough for newer versions with larger blocklists. It doesn't consume much while running, but dynamically it can shoot up rather high, especially for Unbound using blocklists. Try disabling them and see if the crashing changes.

[xavx]

What I've done for both unbound and suricata is to create 2 small bash scripts to monitor their status. If they get killed, they are restarted automatically.

Code Select Expand

root@fwall:~ # cat dnscheck.sh
#!/usr/local/bin/bash

LOGFILE="/root/log/unbound_monitor.log"
TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S")

if ! pgrep -x "unbound" > /dev/null; then
    echo "$TIMESTAMP: Unbound process not found, restarting..." >> $LOGFILE
    /usr/local/sbin/configctl unbound restart
    echo "$TIMESTAMP: Unbound restart command executed" >> $LOGFILE
fi

root@fwall:~ # cat /etc/cron.d/dnscheck
*/5      *       *       *       *    root   /root/dnscheck.sh > /dev/null

A major issue with DNSBL on opnsense is the lack of memory usage optimization when the blocklists get updated.
First issue is the blocklist update process loads all the blocklists in RAM then format them for Unbound, instead of streaming them.
Second issue is at DNSBL swap time, Unbound effectively has 2x all the blocklists loaded for a brief moment before releasing the old blocklist.
Both situations can lead to the highest memory consumer being killed by FreeBSD (expected behavior).

[Patrick M. Hausen]

If you like to use DNSBL I recommend delegating that to a servicd that is designed specifically with large lists in mind. Like AdGuard Home which can run directly on OPNsense and comes with a much nicer UI to manage excepted devices, manual allowlists, statistics etc.

[xavx]

Adguard is indeed the "easiest" way to keep large blocklists without butchering Unbound implementation on Opnsense.
It's unfortunate though that the Adguard plugin isn't officially supported as it's the only viable way.
Unless the Opnsense devs move to RPZ, Unbound will continue to consume 4 times the amount of RAM for DNSBL compared to Adguard.

[franco]

> Second issue is at DNSBL swap time, Unbound effectively has 2x all the blocklists loaded for a brief moment before releasing the old blocklist.

This is a very generic issue that requires intricate implementations to overcome. Alias loading in pf has the exact same issue as an example.


Cheers,
Franco

[Patrick M. Hausen]

@Franco, what are the odds AGH will ever be included as an official community plugin? I mean, there's a FreeBSD port/package, no need to pull obscure binaries from Github or some such. Would be even easier if you would build your ports in Poudriere ;-)

[franco]

Last time Michael checked he said the ports binaries didn't work, but that may have been a long time ago.

Not super comfortable to bring such a thing into the supported ecosystem and people making AI plugins for it rather sooner than later.  There's probably more noise and support questions then.

Other core devs will probably have similar views but we can talk about it at EuroBSDCon.


Cheers,
Franco

[nero355]

4 GB is not enough for newer versions with larger blocklists.

That's INSANE for something as simple as adblocking... :(

Please either listen to Patrick :

If you like to use DNSBL I recommend delegating that to a servicd that is designed specifically with large lists in mind.
Like AdGuard Home which can run directly on OPNsense and comes with a much nicer UI to manage excepted devices, manual allowlists, statistics etc.

Or grab yourself a Raspberry Pi 2B/3B or Intel Atom NUC and run Pi-Hole + Unbound on it :)


_{The laptop I posted this from has 4 GB of RAM FFS! LOL! ^_^}

[notspam]

Perhaps a new approach for dns filter lists in unbound:

Here is the breakdown of how AVM optimized the native DNS filter lists introduced in FRITZ!OS 8.40 / 8.50 to minimize RAM and CPU impact.
------------------------------
## 1. RAM Optimization (Memory Efficiency)

* Highly Compressed Data Structures: AVM does not store the imported text files as raw strings. Instead, during the import process, the FRITZ!Box parses and compiles the domains into highly efficient data structures (likely optimized Tries or compressed Bloom Filters). This reduces the memory footprint to a fraction of the raw file size.
* Low Footprint for Millions of Domains: Real-world testing by the community shows that importing heavy lists like HaGeZi Multi PRO combined with TIF (Malware)—totaling around 2.3 million blocked domains—only increases the RAM usage of an older FRITZ!Box 7590 (which only has 512 MB RAM) by a moderate 5% to 10%. There remains plenty of headroom on all supported models.
* No External Storage Required: Unlike early community speculations, you do not need to connect a USB drive or utilize the internal NAS storage to handle large blocklists. The RAM management handles everything internally.

## 2. CPU and Performance Optimization

* Algorithmic Complexity (O(1) / O(log n)): Thanks to the specialized search trees mentioned above, lookup times are mathematically decoupled from the list's size. Whether a list contains 10,000 or 2.5 million entries, the time it takes to check a domain remains nearly identical. The router never performs a slow sequential string search.
* The "Paradoxical" CPU Relief: User reports from the FRITZ! Labor beta phase revealed that the overall CPU load often decreases during normal web browsing when the filter is active.
* Why this happens: When tracking, advertising, or malware domains are instantly blocked at the DNS stage, the FRITZ!Box never has to establish those TCP/UDP connections, route the packets, or handle Network Address Translation (NAT) for them. Less junk traffic means less work for the CPU.
* Integration with Packet Acceleration: The DNS filtering engine is deeply integrated into FRITZ!OS and operates in tandem with AVM's hardware-level routing accelerators. This ensures that filtering does not bottleneck your overall internet throughput.

In short, AVM has successfully implemented this feature directly into the core of their lightweight Linux-based firmware, making it significantly more resource-efficient than running a heavy Docker container or Pi-hole on an external device.