Messages

From my own testing those regex entries will most likely not work, you should notice that URLs to those domains are now being blocked.
As an example for my regex entries, I went from 2 entries to 12 entries in the whitelist.

This line matched 2 URLs
^\*\.androidtv[a-z]{8}-pa\.googleapis\.com$

This line matched 10 URLs
^\*\.[a-z0-9-]*\.delivery\.roku\.com$

However whitelisting through the unbound reporting page now works correctly, before it was broken. So I believe this is the reason the change was made, but it's unfortunate the fix breaks regex whitelisting functionality.

So going through the github links for the issue/pull request and commit... my understanding is that regex now no longer works and will require a specified domain format such as "subdomain.example.com" in order to get whitelisted successfully.

If so, I think there should have been a more communication in the release notes to indicate that regex whitelisting in unbound will no longer function.

UPDATE
After converting all of my whitelist entries from regex to the actual domain names, the whitelist functions as expected.

i am not sure what this means from the release notes:

unbound: move whitelist (passlist) handling to Unbound plugin

the last time unbound wasn't working right for me after an update. i had to do a full firewall reboot.  i am sure this is different

Not sure exactly, but maybe in doing so it inadvertently changed the processing order of the lists. Also for context I did reboot the firewall on the 25.1.4 update, even though it wasn't required. I have also restarted the unbound service and reapplied the blocklist settings.

Hello,

Since the 25.1.4 update I have noticed that the unbound whitelist (excludes) is now processed after the blocklists in the unbound log, as such it now no longer excludes the regex whitelist entries defined.

Before by processing the whitelist (excludes) before the blocklists resulted in:
blocklist: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.plus.txt (exclude: 17 block: 300615 wildcard: 300615)

Now by processing the whitelist (excludes) after the blocklists results in:
blocklist: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.plus.txt (exclude: 0 block: 298498 wildcard: 298498)

Looks like something needs to be adjusted/fixed here.

So it turns out it was the https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.txt has it down as:

Code Select Expand

*.sentry.io

This is the first time I've had a dns blocking solution (Ubiquiti, PiHole, Netguard, and Adaway) where if the domain is wild card listed in the block list that a specific whitelisted subdomain wouldn't work.

I think this is worth filing a github report as I don't think it should work like this or require regex modification of the entry, what do you think? I'll mostly be repeating your post.

I agree, it seems ridiculous that you have to whitelist the wildcard entry in the blocklist in order for it to work. I don't remember having this problem before, so I think something has changed in the parsing of these lists.

You should be able to block an entire domain and only whitelist the required sub domains.

I see your blocklist says "custom" in the reporting section. I would suggest looking in your blocklist to see what URL(s) are being blocked, then adjust the whitelist rule accordingly. Perhaps it's the parent domain that is being blocked with a wildcard and it is simply matching all subdomains.

Once you identify how it's being blocked in the blocklist, you should more easily be able to create the required whitelist entry.

[Only whitelisting the wildcard entry "*.domain.com" allows the subdomain to resolve, you MUST whitelist the wildcard entry as listed in the blocklist for the desired affect.]

Hello,

I have come across the same behaviour on OPNsense 25.1 using the OISD and Hagezi (Available in 25.1 only) blocklists. I'm not sure if the format described below is blocklist specific.

It appears that they have changed the way blocklists and whitelists are now parsed and whitelisting through the Web UI no longer functions properly as it no longer matches the blocklist entries.

The blocklist entries now all appear to be preceded with

Code Select Expand

*.so, the actual blocklist entry is

Code Select Expand

*.o427061.ingest.sentry.ioBut unblocking through the Web UI only adds

Code Select Expand

o427061.ingest.sentry.iowhich does not match the preceding

Code Select Expand

*.
Hence the Web UI reports that it has been whitelisted but in fact it hasn't as it does not match what's in the blocklist.

I have found using a regular expression is the only way to whitelist the URL(s) successfully, try adding this to your whitelist instead...

Code Select Expand

^\*\.o427061\.ingest\.sentry\.io$
The backslashes are used to escape the characters following the backslash to indicate it should be treated as a literal character. The carat (^) says to match at the beginning of the line and the dollar ($) says to match at the end of the line. This regex ensures that only that URL is matched and does not match another URL by accident. As in regex a star (*) means match any number of preceding characters and a dot (.) means match any character.

Regex Cheatsheet
https://www.rexegg.com/regex-quickstart.php

Test Regex
https://regex101.com/

UPDATE: I have also noticed that if an entire domain in a list is blocked "*.domain.com", then even whitelisting a subdomain "sub.domain.com" also does not work and is still blocked by the wildcard entry in the list. Only whitelisting the wildcard entry "*.domain.com" allows the subdomain to resolve, you MUST whitelist the wildcard entry as listed in the blocklist for the desired affect.

Must be something stuck in the system somewhere. However bufferbloat is no longer that much of a concern for me anymore, so I think I'll just leave stuff disabled for now. I'll probably do a fresh install and a config import at a later date to see if it resolves the issue, but that will be months away.

Anyway thanks for confirming it's just me ;)

Hello,

I just noticed that my Firewall Shaper rule (Using FQ-Codel) for the download pipe is no longer functioning correctly and it cuts the bandwidth greatly. My regular download is about 1600Mbps, and with the download pipe enabled it only max's out around 150Mbps now. The weird thing is that nothing has changed in years in regards to the shaper rules.

FYI, the config is basically as describe in the OPNsense article.
https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html

I tried deleting the pipe, rebooting and re-creating it... same problem. I also upgraded to 25.1-RC1 after taking a snapshot and the issue persists, also changing the bandwidth limit has no effect, it almost seems the limit has been hardcoded. The only thing that works is disabling the download pipe, then I get my max bandwidth again. Also only the download pipe is affected, the upload pipe works as it always has.

I'm not sure when this started, maybe with the update to 24.7.12? not sure.

So for now I am just going to leave the shaper pipes disabled for now, but was wondering has anyone else noticed a bandwidth reduction issue in regards to shaper pipes?

@Franco do you think this is possible? or is there an easy way to incorporate this FreeBSD patch within OPNsense?

I took a look at the FreeBSD patch documentation, but it didn't seem to be as simple as I would like. Also not sure if the FreeBSD patch command would even work correctly in OPNsense.

I was wondering if the following fix could be implemented in to the next point release to resolve this false positive in regards to high cpu interrupt usage status for OPNsense VMs running in XCP-NG.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277231

FYI, It looks like the FreeBSD maintainer that caused all this is trying to work with the upstream dev.
https://github.com/miniupnp/miniupnp/pull/671

However progress only just picked back up again by the looks of it.

It does seem that this miniupnpd package is modified a fair bit from the upstream version from the original developer. I think maybe calling it a port is not really accurate, probably closer to say that it's a fork from the main developer.

Which blurs the lines a bit, but either way I do think that the FreeBSD maintainers seem to be lacking on maintaining their packages, or at least this one in particular.

I'm sure things will get sorted out eventually, it's just a shame on how this bug was handle by the FreeBSD maintainers. Notablely the individual that caused the issue in the past commit and then ultimately fixed it... although in a completely different repository :(

Ya seems to be some favoritism here, I left a new comment on the FreeBSD bugzilla questioning this exact thing. Probably end up getting deleted if they really are playing favorites  ???

OPNsense uses the FreeBSD port of miniupnpd, it is the FreeBSD port maintainers that need to update the package.

Link to the FreeBSD port of miniupnpd:
https://www.freshports.org/net/miniupnpd/

Link to the FreeBSD bugzilla for this issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277226