OPNsense Forum
English Forums => General Discussion => Topic started by: randyrandom on August 26, 2022, 09:14:06 pm
-
Hey,
im using Pi-Hole with over 50 Million Domains to block nearly everything in my Homenetwork, what a "normal User" would call "the Internet".
Any Microsoft Service, any Amazon Service, any Google Service, any Alphabet Service, any Facebook Service, any Tiktok Service, Alibaba, Tencent, Spotify, Nextflix, Paypal, Reddit, and so on. simply everything wich isnt Open Source or known personal data horders.
Every Guest who visits me, know this and know they must use they own mobile data plan if they want to use these services.
But today i found out, that this dont always work. A Guest showed me his new Samsung Phone and after few minutes fideling he suprisingly ask me when i stop blocking facebook/whatsapp. I said i never did.
After a hour of research and experiments, i/we found out that facebook/whatsapp is indeed blocked, if visited by a browser, but it seems that the facebook/whatsapp clients got a update with hardcoded dns into it, wich i cant block because it seems to use DoH (dns over https).
Sure i could export the Pi-Hole blocklists to IP, and block all ~50 Millions IP adresses with opnsense firewall rules. But this is a massive work wich would take me hundreds of years maybe ;D (or is there a function in opnsense where i can import a txt file with ips wich then get blocked?)
Now i search for a way, to redirect any(!) DNS Request to my Pi-hole. Yes, any. DNS, DNSSEC, DoH, what ever exists.
Or maybe you experts know a better way to accomplish that what i want.
Thank you for you help.
-
certainly not the expert here, but take a look at the zenarmor plugin for opnsense.
It can block all the services you mentioned.
And in Adguard home is an option for this also, but i do not use it, so i can not tell if it works as good as with zenarmor.
And there are probably more solutions...
-
Now i search for a way, to redirect any(!) DNS Request to my Pi-hole. Yes, any. DNS, DNSSEC, DoH, what ever exists.
No way.
DoH, DoT or even DoQ (no experience yet with DoQ) cant be redirected to your resolver as the client wont accept the answer of any other DNS server than queried.
The (my) solution is to reject those DNS protocols; most clients / software will fallback to normal DNS which will be redirected; in some cases they wont and will run out of time.
To do this, my FW-rules looks like this:
IPv4 TCP/UDP * * 10.13.12.2 53 (DNS) * * Redirect DNS to this Firewall
IPv6 TCP/UDP * * fd00:10:13:12::acab 53 (DNS) * * Redirect v6 DNS to this Firewall
IPv4+6 TCP ! This Firewall * ! LAN address 853 * * Reject hardcoded DoT-DNS access
IPv4+6 TCP ! This Firewall * DNSServer_merged 443 (HTTPS) * * Reject hardcoded DoH-DNS access
Where the alias "DNSServer_merged" contains some DNS server lists from git and a (very small) DNS server list created by myself.
https://raw.githubusercontent.com/neargle/public-dns-list/master/all.txt
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/iplist.txt
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/ipv6list.txt
https://public-dns.info/nameservers-all.txt
https://raw.githubusercontent.com/trickest/resolvers/main/resolvers.txt
And my own list (not included in mentioned lists when i checked it):
chrome.cloudflare-dns.com
mozilla.cloudflare-dns.com
doh.opendns.com
doh.dns.sb
185.222.222.222
185.184.222.222
2a09::
2a09::1
2a07:a8c0::89:ec71
2a07:a8c1::89:ec71
dns.nextdns.io
-
For sure, the whole of those available DNS-server lists (needed to reject DoH over 443) will never include every single DNS-server, so it still will be possible to override the DoH rule.
Apart from that: DoQ is actually not taken into account in this ruleset.
-
Thanks for the information.
How do you added this lists to the alias?
Simply copy & paste seems not to work for me.
chrome.cloudflare-dns.com
mozilla.cloudflare-dns.com
doh.opendns.com
doh.dns.sb
185.222.222.222
185.184.222.222
2a09::
2a09::1
2a07:a8c0::89:ec71
2a07:a8c1::89:ec71
dns.nextdns.io
if i paste it like that, that whole block is one host.
-
You need to c+p every line seperated ;)
-
I do this more simply thusly:
1) Outbound NAT rules to redirect port 53 TCP/UDP to Pihole (Log to locate devices trying to bypass your DNS and remove them from your network).
2) Outbound NAT rules to redirect port 853 TCP/UDP to Pihole(Log to locate devices trying to bypass your DNS and remove them from your network).
3) Zenarmor tick rule to block DNS over TLS (Zenarmor has a logging interface automatically)
4) Zenarmor tick rule to block DNS over HTTPS
5) LAN rule to block 8853 UDP out (Dont bother logging any chrome browser will trigger log).
6) LAN rule to block 443 UDP out (Dont bother logging any chrome browser will trigger log).
7) LAN AllowList Alias to allow out CND networks if required
8 ) LAN BlockList Alias to block outbound IP's on lists (LOG THIS RULE SO YOU CAN SEE WHEN IP's BLOCKED):
https://raw.githubusercontent.com/dibdot/DoH-IP-blocklists/master/doh-ipv4.txt
https://raw.githubusercontent.com/pallebone/TheGreatWall/master/TheGreatWall_ipv4
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/iplist.txt
https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHipv4.txt
https://raw.githubusercontent.com/cbuijs/accomplist/master/doh/plain.black.ip4cidr.list
List of manual IP's I have found:
Manual added DOH IP's: 1.1.1.1,1.0.0.1,1.1.1.2,1.0.0.2,1.1.1.3,1.0.0.3,104.17.64.4,104.17.65.4,203.107.1.4,193.161.193.99
Manual added DOH ranges: 101.36.166.0/24,203.107.1.0/24
Note : AllowList I have had to open so far contains this port/ip combinations:
Allow out port 443: 151.101.66.133, 151.101.2.133, 151.101.130.133, 172.67.75.103, 104.26.2.13, 185.199.108.153, 185.199.109.153, 185.199.110.153, 185.199.111.153, 104.26.3.13, 151.101.194.133, 216.239.34.21, 44.235.246.155, 151.101.65.195, 104.26.4.174, 172.67.70.80, 104.21.39.13, 172.67.170.203, 151.139.128.10, 104.26.5.174, 151.101.1.195, 141.193.213.21, 172.67.212.2, 104.21.85.239, 44.236.72.93, 104.16.132.229, 141.193.213.20, 217.64.148.8, 104.21.68.104, 96.126.123.244, 45.33.20.235, 44.236.48.31, 216.239.36.21, 216.239.38.21, 104.19.155.92, 104.21.15.239, 167.172.139.120, 216.239.32.21, 90.155.62.13, 90.155.62.14, 95.216.25.250, 162.159.138.85, 162.159.137.85 172.224.62.11 172.224.63.11 172.224.63.19 23.227.38.65
Allow out port 123: 69.1.1.251, 129.250.35.250, 129.250.35.251, 162.248.241.94, 194.36.144.87, 95.216.24.230, 45.76.113.31, 94.16.114.254
Allow out port 80: 151.101.66.133, 151.101.194.133, 151.101.2.133, 151.101.130.133, 141.193.213.20, 172.67.70.80, 104.26.4.174, 184.168.131.241, 17.253.85.204, 162.159.138.85, 162.159.137.85
With this combination I have not been able to find a way to bypass the block unless an IP is added to the allowlist (required if you want to access a site that is a CDN).
I occasionally update this page with new IP's or lists I find (the DOH stuff is near the end):
https://github.com/pallebone/PersonalPiholeListsPAllebone
-
First: I thought im the only stupid guy wich want to block such kind of stuff lol ;D
Glad im not.
Thank you guys for the massive information. As soon i have a Day free where i can risk to be offline, i try it out to implement.
You need to c+p every line seperated ;)
You are kidding me. alone your links you provided, are so long i cant read them all in my left lifetime :o
-
Hey,
im using Pi-Hole with over 50 Million Domains to block nearly everything in my Homenetwork, what a "normal User" would call "the Internet".
Any Microsoft Service, any Amazon Service, any Google Service, any Alphabet Service, any Facebook Service, any Tiktok Service, Alibaba, Tencent, Spotify, Nextflix, Paypal, Reddit, and so on. simply everything wich isnt Open Source or known personal data horders.
Every Guest who visits me, know this and know they must use they own mobile data plan if they want to use these services.
But today i found out, that this dont always work. A Guest showed me his new Samsung Phone and after few minutes fideling he suprisingly ask me when i stop blocking facebook/whatsapp. I said i never did.
After a hour of research and experiments, i/we found out that facebook/whatsapp is indeed blocked, if visited by a browser, but it seems that the facebook/whatsapp clients got a update with hardcoded dns into it, wich i cant block because it seems to use DoH (dns over https).
Sure i could export the Pi-Hole blocklists to IP, and block all ~50 Millions IP adresses with opnsense firewall rules. But this is a massive work wich would take me hundreds of years maybe ;D (or is there a function in opnsense where i can import a txt file with ips wich then get blocked?)
Now i search for a way, to redirect any(!) DNS Request to my Pi-hole. Yes, any. DNS, DNSSEC, DoH, what ever exists.
Or maybe you experts know a better way to accomplish that what i want.
Thank you for you help.
redirect any DNS request to Pi-hole rule (Firewall : NAT : Port Forward):
- interface: your LAN
- proto: tcp/udp
- source: !pi-hole (invert checked)
- ports: any
- destination: !pi-hole (invert checked)
- ports: 53
- redirect target ip: pi-hole
- redirect target port: 53
If you have a choice, maybe take a look at AdGuard Home for adblocking.
AdGuard Home has the option to block services like Facebook with one click.
-
You are kidding me. alone your links you provided, are so long i cant read them all in my left lifetime :o
Only the custom servers needs to be added one by one, the provided lists are added by URL, there ist one alias for every URL and one alias for the custom servers. All those aliases are merged in the "final" alias, used in FW rules:
(https://iili.io/4avxQp.th.png) (https://freeimage.host/i/4avxQp) (https://iili.io/4avojR.th.png) (https://freeimage.host/i/4avojR) (https://iili.io/4avnTv.th.png) (https://freeimage.host/i/4avnTv)
-
If you have a choice, maybe take a look at AdGuard Home for adblocking.
AdGuard Home has the option to block services like Facebook with one click.
Wich most of them dont work (like descriped in my first post), because these only block the access to the website.
If you use a "native" App, like Whatsapp, this dont work anymore because these apps have workarounds builded in.
As example: https://github.com/AdguardTeam/AdGuardHome/issues/1122#issuecomment-550385842
Only the custom servers needs to be added one by one, the provided lists are added by URL, there ist one alias for every URL and one alias for the custom servers. All those aliases are merged in the "final" alias, used in FW rules:
Thanks god. I already got sweaty ;D
Edit:
Oh god is zenarmor great! What a epic plugin. Installed it, and im overwhelmed what options and rules already predefined exist. And the premiumplan for is for under 10€!
And best part is, that already everything works. And that live logging is so clearly arranged and directly in the window operable to block/whitelist etc.
You guys are really heros for me! :)
thats the best part too: https://freeimage.host/i/unbenannt.4N6KLG
Srsly. In few minutes testing, i have already hundreds of different ips and dozens of different countrys where it tried to connect. until it seems it gived up und found a hole. voila, blocked too.
-
I do this more simply thusly:
1) Outbound NAT rules to redirect port 53 TCP/UDP to Pihole (Log to locate devices trying to bypass your DNS and remove them from your network).
2) Outbound NAT rules to redirect port 853 TCP/UDP to Pihole(Log to locate devices trying to bypass your DNS and remove them from your network).
3) Zenarmor tick rule to block DNS over TLS (Zenarmor has a logging interface automatically)
4) Zenarmor tick rule to block DNS over HTTPS
5) LAN rule to block 8853 UDP out (Dont bother logging any chrome browser will trigger log).
6) LAN rule to block 443 UDP out (Dont bother logging any chrome browser will trigger log).
7) LAN AllowList Alias to allow out CND networks if required
8 ) LAN BlockList Alias to block outbound IP's on lists (LOG THIS RULE SO YOU CAN SEE WHEN IP's BLOCKED):
https://raw.githubusercontent.com/dibdot/DoH-IP-blocklists/master/doh-ipv4.txt
https://raw.githubusercontent.com/pallebone/TheGreatWall/master/TheGreatWall_ipv4
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/iplist.txt
https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHipv4.txt
https://raw.githubusercontent.com/cbuijs/accomplist/master/doh/plain.black.ip4cidr.list
List of manual IP's I have found:
Manual added DOH IP's: 1.1.1.1,1.0.0.1,1.1.1.2,1.0.0.2,1.1.1.3,1.0.0.3,104.17.64.4,104.17.65.4,203.107.1.4,193.161.193.99
Manual added DOH ranges: 101.36.166.0/24,203.107.1.0/24
Note : AllowList I have had to open so far contains this port/ip combinations:
Allow out port 443: 151.101.66.133, 151.101.2.133, 151.101.130.133, 172.67.75.103, 104.26.2.13, 185.199.108.153, 185.199.109.153, 185.199.110.153, 185.199.111.153, 104.26.3.13, 151.101.194.133, 216.239.34.21, 44.235.246.155, 151.101.65.195, 104.26.4.174, 172.67.70.80, 104.21.39.13, 172.67.170.203, 151.139.128.10, 104.26.5.174, 151.101.1.195, 141.193.213.21, 172.67.212.2, 104.21.85.239, 44.236.72.93, 104.16.132.229, 141.193.213.20, 217.64.148.8, 104.21.68.104, 96.126.123.244, 45.33.20.235, 44.236.48.31, 216.239.36.21, 216.239.38.21, 104.19.155.92, 104.21.15.239, 167.172.139.120, 216.239.32.21, 90.155.62.13, 90.155.62.14, 95.216.25.250, 162.159.138.85, 162.159.137.85 172.224.62.11 172.224.63.11 172.224.63.19 23.227.38.65
Allow out port 123: 69.1.1.251, 129.250.35.250, 129.250.35.251, 162.248.241.94, 194.36.144.87, 95.216.24.230, 45.76.113.31, 94.16.114.254
Allow out port 80: 151.101.66.133, 151.101.194.133, 151.101.2.133, 151.101.130.133, 141.193.213.20, 172.67.70.80, 104.26.4.174, 184.168.131.241, 17.253.85.204, 162.159.138.85, 162.159.137.85
With this combination I have not been able to find a way to bypass the block unless an IP is added to the allowlist (required if you want to access a site that is a CDN).
I occasionally update this page with new IP's or lists I find (the DOH stuff is near the end):
https://github.com/pallebone/PersonalPiholeListsPAllebone
Somehow i cant create a NAT Outbound rule to specific my pi-hole (see attachment).
It changed everytime to 192.168.5.0 as destination.
Edit: Ok i followed this guide: https://homenetworkguy.com/how-to/redirect-all-dns-requests-to-local-dns-resolver/
Wich works technically, but practical not. Any normal DNS request gets redirected to pi-hole.
BUT, after the dns query reached pi-hole, pihole start a request to other upstream DNS Server (level3 in this case, for testing), wich gets directly redirected to pi-hole again, where the loop starts from new.
-
Ok, after hours of fideling im now frustrated and need a break.
After i found out, that zenarmor works, but sadly not to 100% (because i think they use tables in the background too wich lack behind actual used ips or so?) i tried simply to create a floating rule with a alias.
The IPs i need, i got from https://github.com/NetSPI/NetblockTool.
For Facebook for exmaple: https://pastebin.com/raw/VTfsA4DP
Created the Rule like here described: https://www.allthingstech.ch/using-opnsense-and-ip-blocklists-to-block-malicious-traffic
And? It didnt work. In the Liveview from the Firewall, i can see he allows connection to 157.240.196.111 for example. but it is definitly blocked (or should) by 157.240.196.0/24 (wich would be 0-255).
-
Ok, after hours of fideling im now frustrated and need a break.
After i found out, that zenarmor works, but sadly not to 100%
I'm not sure what you're trying to achieve exactly, but ultimately if you let an App access its own servers on the wider internet, then ultimately you wont be able to block them doing DNS lookups if they use their own servers to do so, which some do!
While many DOH clients use public DNS servers such as Google, which makes it easier to firewall their well known addresses; nothing stops apps from running their own DOH servers on their own infrastructure. And by design DOH traffic is indistinguishable from regular web traffic.
So I don't think a 100% solution is possible unless you completely firewall all of the apps servers and thus disable the app completely.
-
im trying to achieve to block every thing i mentioned on my first post.
In the meantime, i created 4 VM's with there own network.
1x opnsense
1x pihole
2x clients
There i get the same problem. As soon the pi-hole is in the same network like the others, the "dns override" gets in a loop. If the Pi-hole is on a another network, it works.
like described here: https://forum.opnsense.org/index.php?topic=30066.msg145392#msg145392
-
im trying to achieve to block every thing i mentioned on my first post.
Right. Which included "and so on".
In any event, your list is so expansive as to effectively cover a large and hugely diverse range of applications and services, thus why I was asking for something more along the line of categorization or generalization that might be applicable.
Even just looking at your specific huge list, you do realise that these sort of Internet giants are spinning up new IP addresses daily as they deploy and expand their networks.
-
Yes, i know.
But for now, i have a Solution for me.
i tried this: https://www.allthingstech.ch/using-opnsense-and-ip-blocklists-to-block-malicious-traffic last week, but didnt worked for me, because i did a mistake. it seems opnsense didnt accept 12.9.69.72/29 for example. It only accepts ips.
So i created a small VM with Debian on my server. There now runs a cronjob every 15 Minutes wich runs https://github.com/NetSPI/NetblockTool
For example: python3 NetblockTool.py -v Facebook -s -4
Wich will you get all IP's (v4, because v6 is deactivated anyway) wich belongs to Facebook and subsidiaries of it.
In my case:
[*] Getting subsidiary information for Facebook
[*] Gathering company information for Facebook from EDGAR database
[*] Gathering company documents for Facebook from EDGAR database
[*] Status: 1/5
[*] Status: 2/5
[*] Status: 3/5
[*] Status: 4/5
[*] Status: 5/5
[*] Removed companies with no document information, 1/5 remain
[*] Getting list of Facebook subsidiaries
[*] Searching filings for EX-21 documents
[*] Found: https://www.sec.gov/Archives/edgar/data/1326801/000132680122000018/0001326801-22-000018-index.htm
[*] Downloading EX-21 document
[*] Found: https://www.sec.gov/Archives/edgar/data/1326801/000132680122000018/fb-12312021x10kexhibit211.htm
[*] Parsing subsidiaries
[*] Found 26 subsidiaries
[*] Cassin Networks ApS
[*] Edge Network Services Limited
[*] FCL Tech Limited
[*] Facebook Holdings LLC
[*] Facebook Operations LLC
[*] Facebook Payments Inc
[*] Facebook Technologies LLC
[*] Facebook UK Limited
[*] Greater Kudu LLC
[*] Hibiscus Properties LLC
[*] Instagram LLC
[*] META PLATFORMS INC
[*] Meta Platforms Ireland Limited
[*] Morning Hornet LLC
[*] Novi Financial Inc
[*] Pinnacle Sweden AB
[*] Raven Northbrook LLC
[*] Runways Information Services Limited
[*] Scout Development LLC
[*] Siculus Inc
[*] Sidecat LLC
[*] Stadion LLC
[*] Starbelt LLC
[*] Vitesse LLC
[*] WhatsApp LLC
[*] Winner LLC
The result you get is a csv file with all IP Ranges. Then i found this: https://stackoverflow.com/questions/16986879/bash-script-to-list-all-ips-in-prefix
With that i can let calculate all ips from these ranges with ./script -i input.txt > output.txt.
The output.txt is on a local nginx server and my opnsense loads that.
And this seems to work. Now i have since ~1 Hour Live View open and can see how Whatsapp tries to connect tousands of different ips every secound (and in the app it still tries to send/recive the messages). Without that blocklist (only zenarmor), it tries for ~10 minutes too and after that time it had success and found a connection wich works. Now not more :-)
Next week i write a script to automate that and to create cronjobs for the other services like Microsoft, Amazon and others too.
And i need to find out why i cant redirect all DNS Request to my Pi-hole without it gets looped endless.