Blocking ads using only unbound

[azfirefighter]

I managed to get ad blocking done using only Unbound.  Our command line friends, curl and awk do the heavy lifting.

This post is a little messy right now, but I'll get it cleaned up a little later.

HOUSEKEEPING INFO:
You will need to be able to log into the machine using either the console or SSH to get this accomplished.

The list I'm using is Steven Black's "Unified+Gambling+Fake News".  You can choose from any of his other lists at https://github.com/StevenBlack/hosts/blob/master/readme.md.  Rather than using one of the "Raw Hosts" URLS, choose one of the "Non Github Mirror" URLS.  Curl seems to have an issue with the "Raw Hosts" URLS.

GETTING STARTED:

• Make sure you've got a fully working Unbound setup.
• Then, log in via the command line, choose #8 and create a directory to house your working files.
• If you want to use the script as-is, then use "/usr/share/blocklist"
• Place the following script in the directory and name it "getlist.sh" (also as attachment)

If you've opted to use a different filter list, you'll have to change the address that curl is using in the following script.


#!/bin/sh
#
# First, let's get the list

curl http://sbc.io/hosts/alternates/fakenews-gambling/hosts -o blockhosts.txt

# Too bad it's got a lot of stuff at the beginning we don't need.
# Time to remove the cruft

awk 'NR>=35{ print }' blockhosts.txt > hosts-fixed.txt

# Time to make it usable for the unbound service

echo -n "server:" > block.conf # MUST HAVE THIS AT THE TOP OF THE FILE

cat hosts-fixed.txt | grep '^0\.0\.0\.0' | awk '{ print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\"" }' >> block.conf

# Present it to unbound
chown unbound:unbound block.conf
mv block.conf /var/unbound/block.conf
unbound-control -c /var/unbound/unbound.conf reload

• Make the script executable: chmod u+x /usr/share/blocklist/getlist.sh
• You must run the script ONCE from the command line or this will fail: /usr/share/blocklist/getlist.sh
• ** Hint: don't log out just yet....

Here's where you add the block list to Unbound.

• Open your browser and log into the opnsense page.
• Navigate to Services -> Unbound DNS -> General (https://<your_machine>/services_unbound.php)
• Scroll down the "General" page until you see the button for Advanced and click on it.
• In the "Custom Options" box, enter: include: /var/unbound/block.conf
• Save your changes and then apply them.

Provided everything was entered correctly, the list should now be applied.

Since the "Cron" opnsense page doesn't allow for custom actions, you'll have to head back to the command line.
I've opted to update the list every night at 23:00 local.

If you're still logged in, keep going.  Otherwise, log back into the command line and choose option #8.

Then:

• Type crontab -e, press Enter and go to the end of the file (you may have to hit the "End" key on the last line)
• Press a
• Press ENTER
• Type: 0     23     *     *     *     (/usr/share/blocklist/getlist.sh) > /dev/null
• Press ESC, then : and finally wq!

Your system should now update the list, mangle it into a format that Unbound can use and reload the list every night at 2300 (local).

I'm working on making a command line install script to make the process a little easier for inexperienced people.  Please be patient.

[bigeazy000]

This is great, thanks for sharing! I was waiting for UnboundBL to materialize, but this will work nicely instead..

[andy]

I'll be keeping an eye on this--would love to get a blacklist feature working

[Serius]

Thanks!

[mimugmail]

https://github.com/opnsense/plugins/pull/1495

[Pranjal]

Hey buddy I saw your post of blocking ad by unbound I applied same but not working ....plzz tell me how to set up the unbound ...moreover by this I can block my own define websites or not??

[mimugmail]

Which post do you mean?

[Pranjal]

Leave that..sir....just tell me that how to block domain aur websites in opn sense....I tried your app detection method in IPS but by that I can only block limited sites...if I want to block any other perticular site like Wikipedia...Amazon so how I can do it..

[mimugmail]

Transparent Proxy and blacklisted sites.

[Pranjal]

Sir I want to know how to use black listed sites I tried my best but can't able to do show...plz can u tell me step by step
Please

[mimugmail]

https://docs.opnsense.org/manual/how-tos/proxytransparent.html

With option to only log SNI, then you can use the blacklist feature in access control

[Pranjal]

Ok thanku sir I will try this....by the way how enable this SNI option

[mimugmail]

Just follow the guide, you will see it :)

[Pranjal]

Thanku so much...I will try it ...but I want your support also...I want to learn this firewall ...kepp supporting sir

[Pranjal]

Hello again...I tried this method and I was successful ...but still I need one help ...first of all tell me why we are using transperant proxy in this web filtering what is the role of that...means I am able to block the domain by web filter without use of transparent proxy...so what is the use of transparent proxy please tell me