I have been experimenting with the Unbound Block List (DEFAULT and USER DEFINED) functionality.
It appears that the DEFAULT list removes duplicate domain and domain pattern entries.
Example: If I select all OPNsense DEFAULT options, the logs indicate it loads over 6M individual entries and coalesces it down to @ 3.5M unique entries. Unfortunately, this final list does not appear to be written to disk for final inspection.
It is not clear if the same thing happens with the USER DEFINED lists and it appears it does NOT remove duplicates between the DEFAULT and USER DEFINED lists.
Example: If I add some of the same lists that are in the DEFAULT lists into the USER DEFINED LISTS, the final aggregate number increases (should NOT increase because they are identical duplicate lists, URLs are identical).
A simple strcmp() of the list URLs would eliminate such duplicates along with a possible warning log for the user to clean up his duplicate main lists.
Also, it would be nice if both DEFAULT and USER LISTS would have granular duplicate removals at the domain and domain pattern levels (e.g. sort -u).
FWIW, here are the URLs for the current DEFAULT block lists (generated from the "Services > Unbound DNS > Blocklist > Type of DNSBL" pulldown check list) in version 25.1.6_4-amd64.
Compare them against your USER DEFINED lists to avoid duplicates.
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&mimetype=plaintext (block: 3440 wildcard: 0)
https://adaway.org/hosts.txt (block: 6540 wildcard: 0)
https://big.oisd.nl/domainswild (block: 177807 wildcard: 177807)
https://blocklistproject.github.io/Lists/alt-version/abuse-nl.txt (block: 435144 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/ads-nl.txt (block: 154554 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/crypto-nl.txt (block: 23758 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/drugs-nl.txt (block: 26031 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/facebook-nl.txt (block: 22459 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/fraud-nl.txt (block: 196082 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/gambling-nl.txt (block: 2500 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/malware-nl.txt (block: 435214 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/phishing-nl.txt (block: 190221 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/piracy-nl.txt (block: 2153 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/porn-nl.txt (block: 500281 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/ransomware-nl.txt (block: 1904 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/redirect-nl.txt (block: 108684 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/scam-nl.txt (block: 1274 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/tiktok-nl.txt (block: 3699 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/torrent-nl.txt (block: 2623 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/tracking-nl.txt (block: 15070 wildcard: 0)
https://blocklistproject.github.io/Lists/alt-version/youtube-nl.txt (block: 24280 wildcard: 0)
https://nsfw.oisd.nl/domainswild (block: 447829 wildcard: 447829)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/anti.piracy.txt (block: 10108 wildcard: 10108)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/doh-vpn-proxy-bypass.txt (block: 4302 wildcard: 4302)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/dyndns.txt (block: 1391 wildcard: 1391)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/fake.txt (block: 11450 wildcard: 11450)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/gambling.medium.txt (block: 281251 wildcard: 281251)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/gambling.mini.txt (block: 129936 wildcard: 129936)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/gambling.txt (block: 565695 wildcard: 565695)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/hoster.txt (block: 2110 wildcard: 2110)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/light.txt (block: 42223 wildcard: 42223)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/multi.txt (block: 153900 wildcard: 153900)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/nosafesearch.txt (block: 183 wildcard: 183)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/popupads.txt (block: 102841 wildcard: 102841)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.mini.txt (block: 54878 wildcard: 54878)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.plus.mini.txt (block: 68301 wildcard: 68301)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.plus.txt (block: 303359 wildcard: 303359)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.txt (block: 213448 wildcard: 213448)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/tif.medium.txt (block: 320047 wildcard: 320047)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/tif.mini.txt (block: 83140 wildcard: 83140)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/tif.txt (block: 655058 wildcard: 655058)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/ultimate.mini.txt (block: 81657 wildcard: 81657)
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/ultimate.txt (block: 325213 wildcard: 325213)
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (block: 170009 wildcard: 0)
https://small.oisd.nl/domainswild (block: 43527 wildcard: 43527)
https://threatfox.abuse.ch/downloads/hostfile (block: 65621 wildcard: 1)
https://v.firebog.net/hosts/AdguardDNS.txt (block: 106312 wildcard: 0)
https://v.firebog.net/hosts/Easylist.txt (block: 27916 wildcard: 0)
https://v.firebog.net/hosts/Easyprivacy.txt (block: 42361 wildcard: 0)