Some domains pass Squid, Unbound, OpenDNS trio

[eprom]

Hi Friends,
I am using opnsense in my little shop, but have some leak problems.

System:
OpenDNS as system wide DNS server (enabled "Do not use the local DNS service as a nameserver for this system")
Unbound ("Forwarding enabled" if not enabled things go worser)
Squid (Transparent, ACL Lists added)
IPv6 Disabled (on Firewall,DHCP, On Squid use IPv4 first enabled)

To test my lists working or not I have used "Xenu Link Checker" and start a test with "dsi.ut-capitole.fr" pfsense optimizied lists.

After test with ~22000 urls, I got ~500 reachable urls. And wierd things happening.
All the leaked urls were in squid ACL
* Most of the urls blocked by OpenDNS (but 500 passed "OK its possible")
* Squid cannot catch this 500 urls they are also in ACL

How a trick these sites are using to leak?
Some on cloudflare I blocked all IP Ranges for it on Firewall
But I see lots of other host/name server can trick like this. It is not a solution to block hosts IP Range, lots of clean sites affected from this.

Thanks for advance and help,

PS: I can add leaked urls but all are porn sites so I dont want to add. If needed I can add.

[eprom]

At least someone can test/control the domains I can not block, Selected non porn ones.

These are all in my Squid ACL to block, but squid not catch them

Code: [Select]

http://www.camellist.com/index.html
http://www.aimee-sweet.com/
http://amateur-invest.com/
http://blackonslut.com/
https://brastart.com/
http://celebdb.com/
http://cheat.com/
http://rlddirect.com/

[eprom]

http://buggy-breast.com/ returns as http://back.arthydate.com/

How a redirect trick is this, How they trick this secure DNS servers (OpenDNS, ClearBrowsing "tried both")

[eprom]

at The END my conclusion about this problem, to help others.

1-) Squid asking urls to DNS whether it is in block list or not
2-) Unbound use root DNS's if not forwarded so it is not safe
3-) DNS are not 100% trustable also OpenDNS or ClearBrowsing
(For adult filtering OpenDNS is 85% safe, Clearbrowsing is 95% safe)

What to do:
I have transferred all my Squid ACL's to Unbound blocklist by help of this thread https://forum.opnsense.org/index.php?topic=13466.0
So response times are better than ever. More trustable.

I am using Squid as logger
I am using OpenDNS/ClearBrowsing as my Unbound Forwarder
I am using Firewall GeoIP block for China like destinations

Wish helps to understand working diagram of opensense for newbie user like me

and a big thanks to OPNSense and Community Forum Users

EDIT (More findings):

1- Using Unbound to block urls help theese things,
* Have a better DNS response time, especially ad traffic is very high so if you wait squid to block them lots of dns and blocking page traffic occurs.
* access.log for squid is smaller now beause urls blocked by Unbound does not writing to log. (firewall block written on access.log)
* Unbound blocks redirects (if in its block list), But on only squid setup Squid asking to dns and if the answer is redirected to new url squid cannot block it.

2- Block all the dns (port 53) traffic except internal dns (explained how by https://homenetworkguy.com/how-to/configure-opnsense-firewall-rules/)

[eprom]

Things going more crazy way. This is a cat and mouse game.

When I was thinking everything is fine (on my last report) I found a new game. DNS over HTTPS which is nearly going standart.

***- When played with firefox and set DoH on, filtering on DNS side gone so Start more complex solutions (maybe because of my basic level skills).
From now on
1- Get the unbound host lists copy and set it as ACL on squid again (now Squid and Unbound has same block lists running)
2- Collect the lists of all dns servers around world and block all of them on firewall side (except my used dns resolvers)

But i can not relaxed, I am still curious

PS: This thread became a blog like self-speaking self-listening page

[Antaris]

Dude, i feel you... There is more up-to-date non-DNS method of filtration called Sensei here:
https://forum.opnsense.org/index.php?topic=9521.0
It's way wider, faster and easier to use it. The free version works at glance way better in any means.

[eprom]

Thanks for the tip for new method,
I am on it and will give it a try.

Cheers,

Dude, i feel you... There is more up-to-date non-DNS method of filtration called Sensei here:
https://forum.opnsense.org/index.php?topic=9521.0
It's way wider, faster and easier to use it. The free version works at glance way better in any means.

[eprom]

http://buggy-breast.com/ returns as http://back.arthydate.com/

How a redirect trick is this, How they trick this secure DNS servers (OpenDNS, ClearBrowsing "tried both")

Found the answe after two months,

This trick is named CNAME Cloaking, Using AdGuard DNS on Unbound for blocking now and everything fine for now.