25.1.4 update breaks unbound whitelist

[xpendable]

Hello,

Since the 25.1.4 update I have noticed that the unbound whitelist (excludes) is now processed after the blocklists in the unbound log, as such it now no longer excludes the regex whitelist entries defined.

Before by processing the whitelist (excludes) before the blocklists resulted in:
blocklist: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.plus.txt (exclude: 17 block: 300615 wildcard: 300615)

Now by processing the whitelist (excludes) after the blocklists results in:
blocklist: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.plus.txt (exclude: 0 block: 298498 wildcard: 298498)

Looks like something needs to be adjusted/fixed here.

[DEC670airp414user]

i am not sure what this means from the release notes:

unbound: move whitelist (passlist) handling to Unbound plugin

the last time unbound wasn't working right for me after an update. i had to do a full firewall reboot.  i am sure this is different

[xpendable]

i am not sure what this means from the release notes:

unbound: move whitelist (passlist) handling to Unbound plugin

the last time unbound wasn't working right for me after an update. i had to do a full firewall reboot.  i am sure this is different

Not sure exactly, but maybe in doing so it inadvertently changed the processing order of the lists. Also for context I did reboot the firewall on the 25.1.4 update, even though it wasn't required. I have also restarted the unbound service and reapplied the blocklist settings.

[franco]

Context:

https://github.com/opnsense/core/issues/8273
https://github.com/opnsense/core/pull/8415
https://github.com/opnsense/core/commit/aa2cff3e665fd

[xpendable]

So going through the github links for the issue/pull request and commit... my understanding is that regex now no longer works and will require a specified domain format such as "subdomain.example.com" in order to get whitelisted successfully.

If so, I think there should have been a more communication in the release notes to indicate that regex whitelisting in unbound will no longer function.

UPDATE
After converting all of my whitelist entries from regex to the actual domain names, the whitelist functions as expected.

[doktornotor]

Sigh. So, what is the deal with this change? Regex no longer works at all? "Converting all of my whitelist entries from regex to the actual domain names" is not possible / practical in some use cases, that is the whole point of using the regex in the first place. Or, what's "actual domain name"? Examples - do these still work or not?

Code Select Expand

(.*)?(\.)?akamai.net
(.*)?(\.)?akamaiedge.net
(.*)?(\.)?edgekey.net
(.*)?(\.)?downloads.hpe.com
(.*)?(\.)?gslb-downloads-hpe-com.ext.hpe.com
(.*)?(\.)?gslb-downloads-hpe-com.glb1.hpe.com
(.*)?(\.)?gstatic.com
(.*)?(\.)?api2.branch.io
(.*)?(\.)?cdn.branch.io


[xpendable]

From my own testing those regex entries will most likely not work, you should notice that URLs to those domains are now being blocked.
As an example for my regex entries, I went from 2 entries to 12 entries in the whitelist.

This line matched 2 URLs
^\*\.androidtv[a-z]{8}-pa\.googleapis\.com$

This line matched 10 URLs
^\*\.[a-z0-9-]*\.delivery\.roku\.com$

However whitelisting through the unbound reporting page now works correctly, before it was broken. So I believe this is the reason the change was made, but it's unfortunate the fix breaks regex whitelisting functionality.