I'm trying to figure out why my DEC850 running 24.10.1 is not respecting all the entries on the whitelist.
The blacklist blocking is working fine as far as I can tell but I've added
o427061.ingest.sentry.io to the whitelist so my HAOS setup and send diagnostic report back but one of my blacklist sources is blocking it. I've tried rebooting the router, waiting 2 days, restarting unbound service, and verified that Unbound is set to 'Flush DNS Cache during reload' as indicated in other threads similar to this.
QuoteReporting: Unbound DNS
(https://i.ibb.co/35p8NXZg/image.png) (https://ibb.co/hxBnW5z3)
I've also checked in
QuoteInterfaces: Diagnostics: DNS Lookup
by putting in that it is being blocked. I also can not ping it from a system behind the router and HAOS is giving errors about not being able to reach it.
(https://i.ibb.co/ccHRKPdm/image.png) (https://ibb.co/xS0wKyQ4)
However, when I put in a DNS server in the diagnostics area I get a return and it does not show a CNAME so that isn't the issue.
(https://i.ibb.co/6cSGQkkk/image.png) (https://ibb.co/NgqfRMMM)
I assume I've misconfigured something. Unbound is active (and working for everything else but the whitelist). I have it in recursive mode (
QuoteSystem: Settings: General
has no DNS servers listed, Unbound does not have any DNS over TLS servers enabled, no DNS query forwarding, DNS server options unchecked) I also have DNSSEC support enabled.
(https://i.ibb.co/nskfXRCw/image.png) (https://ibb.co/jv4GCHR8)
Can anyone point me to other places I should check or additional information to figure out what I have wrong?
So I'm getting rather desperate to find a solution to this. I can't see that rebuilding would help as I'd end up at the same configuration end point. (although if you want details to point out where I have misconfigured and broke something I am happy to provide more)
Does anyone thing that wiping my OpnSense install on the DEC850 and restoring from a backup might help?
Hello,
I have come across the same behaviour on OPNsense 25.1 using the OISD and Hagezi (Available in 25.1 only) blocklists. I'm not sure if the format described below is blocklist specific.
It appears that they have changed the way blocklists and whitelists are now parsed and whitelisting through the Web UI no longer functions properly as it no longer matches the blocklist entries.
The blocklist entries now all appear to be preceded with
*.so, the actual blocklist entry is
*.o427061.ingest.sentry.ioBut unblocking through the Web UI only adds
o427061.ingest.sentry.iowhich does not match the preceding
*.
Hence the Web UI reports that it has been whitelisted but in fact it hasn't as it does not match what's in the blocklist.
I have found using a regular expression is the only way to whitelist the URL(s) successfully, try adding this to your whitelist instead...
^\*\.o427061\.ingest\.sentry\.io$
The backslashes are used to escape the characters following the backslash to indicate it should be treated as a literal character. The carat (^) says to match at the beginning of the line and the dollar ($) says to match at the end of the line. This regex ensures that only that URL is matched and does not match another URL by accident. As in regex a star (*) means match any number of preceding characters and a dot (.) means match any character.
Regex Cheatsheet
https://www.rexegg.com/regex-quickstart.php
Test Regex
https://regex101.com/
UPDATE: I have also noticed that if an entire domain in a list is blocked "*.domain.com", then even whitelisting a subdomain "sub.domain.com" also does not work and is still blocked by the wildcard entry in the list. Only whitelisting the wildcard entry "*.domain.com" allows the subdomain to resolve, you MUST whitelist the wildcard entry as listed in the blocklist for the desired affect.
Quote from: xpendable on February 04, 2025, 04:11:08 PMHello,
I have come across the same behaviour on OPNsense 25.1 using the OISD and Hagezi (Available in 25.1 only) blocklists. I'm not sure if the format described below is blocklist specific.
It appears that they have changed the way blocklists and whitelists are now parsed and whitelisting through the Web UI no longer functions properly as it no longer matches the blocklist entries.
The blocklist entries now all appear to be preceded with
*.so, the actual blocklist entry is
*.o427061.ingest.sentry.ioBut unblocking through the Web UI only adds
o427061.ingest.sentry.iowhich does not match the preceding *.
Hence the Web UI reports that it has been whitelisted but in fact it hasn't as it does not match what's in the blocklist.
I have found using a regular expression is the only way to whitelist the URL(s) successfully, try adding this to your whitelist instead...
^\*\.o427061\.ingest\.sentry\.io$
The backslashes are used to escape the characters following the backslash to indicate it should be treated as a literal character. The carat (^) says to match at the beginning of the line and the dollar ($) says to match at the end of the line. This regex ensures that only that URL is matched and does not match another URL by accident. As in regex a star (*) means match any number of preceding characters and a dot (.) means match any character.
Regex Cheatsheet
https://www.rexegg.com/regex-quickstart.php
Test Regex
https://regex101.com/
UPDATE: I have also noticed that if an entire domain in a list is blocked "*.domain.com", then even whitelisting a subdomain "sub.domain.com" also does not work and is still blocked by the wildcard entry in the list. Only whitelisting the wildcard entry "*.domain.com" allows the subdomain to resolve.
Thank you! That seems like a solid lead and I wonder why the change isn't documented?
I did give it a try (restarted unbound after changing the entry and waited 2 hours) but it didn't seem to work.
(https://i.ibb.co/SbfsT4F/image.png) (https://ibb.co/T6WvXx9)
(https://i.ibb.co/gF46Hq9Q/image.png) (https://ibb.co/wrcY3Gg8)
(https://i.ibb.co/ycCnR0rt/image.png) (https://ibb.co/DHFDb4Zh)
(https://i.ibb.co/cX7N8dXs/image.png) (https://ibb.co/x8RMgQ8r)
Is there a way to see in better detail how it is being blocked?
I also notice that with this regex entry that in the 'Reporting: Unbound DNS' page that the entry is showing the button to denote that it isn't on the white list?
I see your blocklist says "custom" in the reporting section. I would suggest looking in your blocklist to see what URL(s) are being blocked, then adjust the whitelist rule accordingly. Perhaps it's the parent domain that is being blocked with a wildcard and it is simply matching all subdomains.
Once you identify how it's being blocked in the blocklist, you should more easily be able to create the required whitelist entry.
Quote from: xpendable on February 05, 2025, 05:18:54 PMI see your blocklist says "custom" in the reporting section. I would suggest looking in your blocklist to see what URL(s) are being blocked, then adjust the whitelist rule accordingly. Perhaps it's the parent domain that is being blocked with a wildcard and it is simply matching all subdomains.
Once you identify how it's being blocked in the blocklist, you should more easily be able to create the required whitelist entry.
So it turns out it was the https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.txt has it down as:
*.sentry.io This is the first time I've had a dns blocking solution (Ubiquiti, PiHole, Netguard, and Adaway) where if the domain is wild card listed in the block list that a specific whitelisted subdomain wouldn't work.
I think this is worth filing a github report as I don't think it should work like this or require regex modification of the entry, what do you think? I'll mostly be repeating your post.
Quote from: charles.adams on February 06, 2025, 12:24:11 AMSo it turns out it was the https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro.txt has it down as:
*.sentry.io
This is the first time I've had a dns blocking solution (Ubiquiti, PiHole, Netguard, and Adaway) where if the domain is wild card listed in the block list that a specific whitelisted subdomain wouldn't work.
I think this is worth filing a github report as I don't think it should work like this or require regex modification of the entry, what do you think? I'll mostly be repeating your post.
I agree, it seems ridiculous that you have to whitelist the wildcard entry in the blocklist in order for it to work. I don't remember having this problem before, so I think something has changed in the parsing of these lists.
You should be able to block an entire domain and only whitelist the required sub domains.