OPNsense Forum
Archive => 20.7 Legacy Series => Topic started by: sjjh on December 05, 2020, 10:21:23 pm
-
We're using squid with some categories of the UT1 Blacklist (https://dsi.ut-capitole.fr/blacklists/index_en.php) to filter the web traffic. I'm trying to understand why squid is blocking the domain https://livingsoilssymposium.ca/ (https://livingsoilssymposium.ca/) (IP: 107.180.24.240). I get the error message
The following error was encountered while trying to retrieve the URL: https://107.180.24.240/*
Access Denied.
Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.
when trying to open the url. The remote blacklist is configured as Blacklist1. If I search for the domain or IP address, I don't get any hits:
$ sudo cat /usr/local/etc/squid/acl/Blacklist1 | grep "livingsoilssymposium.ca"
$ sudo cat /usr/local/etc/squid/acl/Blacklist1 | grep "107.180.24.240"
$
I enabled the squid debugging config as described in the thread https://forum.opnsense.org/index.php?topic=20204.msg93533
I belive, this is the relevant excerpt (from OPNsense web GUI, filtered by the IP address):
2020-12-05T21:50:05 squid .066 kid1| 33,2| client_side.cc(586) swanSong: local=107.180.24.240:443 remote=10.63.19.139:59058 flags=33
2020-12-05T21:50:05 squid .066 kid1| 33,2| client_side.cc(895) kick: local=107.180.24.240:443 remote=10.63.19.139:59058 flags=33 Connection was closed
2020-12-05T21:50:05 squid .057 kid1| 28,3| DomainData.cc(115) match: aclMatchDomainList: 'ip-107-180-24-240.ip.secureserver.net' found
2020-12-05T21:50:05 squid .057 kid1| 28,3| DomainData.cc(110) match: aclMatchDomainList: checking 'ip-107-180-24-240.ip.secureserver.net'
2020-12-05T21:50:05 squid .057 kid1| 28,3| DomainData.cc(115) match: aclMatchDomainList: '107.180.24.240' NOT found
2020-12-05T21:50:05 squid .057 kid1| 28,3| DomainData.cc(110) match: aclMatchDomainList: checking '107.180.24.240'
2020-12-05T21:50:05 squid .057 kid1| 28,3| RegexData.cc(43) match: checking '107.180.24.240:443'
2020-12-05T21:50:05 squid .057 kid1| 28,3| RegexData.cc(43) match: checking '107.180.24.240:443'
2020-12-05T21:50:05 squid .056 kid1| 33,2| client_side.cc(2742) httpsSslBumpAccessCheckDone: sslBump action peekneeded for local=107.180.24.240:443 remote=10.63.19.139:59058 FD 1097 flags=33
2020-12-05T21:50:04 squid .565 kid1| 33,2| client_side.cc(586) swanSong: local=107.180.24.240:443 remote=10.63.19.139:59056 flags=33
2020-12-05T21:50:04 squid .565 kid1| 33,2| client_side.cc(895) kick: local=107.180.24.240:443 remote=10.63.19.139:59056 flags=33 Connection was closed
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(115) match: aclMatchDomainList: 'ip-107-180-24-240.ip.secureserver.net' found
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(110) match: aclMatchDomainList: checking 'ip-107-180-24-240.ip.secureserver.net'
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(115) match: aclMatchDomainList: '107.180.24.240' NOT found
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(110) match: aclMatchDomainList: checking '107.180.24.240'
2020-12-05T21:50:04 squid .546 kid1| 28,3| DestinationDomain.cc(96) match: Can't yet compare 'remoteblacklist_Blacklist1' ACL for 107.180.24.240
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(115) match: aclMatchDomainList: '107.180.24.240' NOT found
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(110) match: aclMatchDomainList: checking '107.180.24.240'
2020-12-05T21:50:04 squid .546 kid1| 28,3| RegexData.cc(43) match: checking '107.180.24.240:443'
2020-12-05T21:50:04 squid .546 kid1| 28,3| RegexData.cc(43) match: checking '107.180.24.240:443'
2020-12-05T21:50:04 squid .546 kid1| 33,2| client_side.cc(2742) httpsSslBumpAccessCheckDone: sslBump action peekneeded for local=107.180.24.240:443 remote=10.63.19.139:59056 FD 1048 flags=33
So in the end the connection gets closed, and before some match was found with a variant of the IP address as subdomain? Why is squid checking for this IP-related subdomain? And where's that match coming from? I do see following hits in the black list:
$ sudo cat /usr/local/etc/squid/acl/Blacklist1 | grep "secureserver.net"
.ip.secureserver.net
.phx3.secureserver.net
.ams3.secureserver.net
.sin3.secureserver.net
.iad2.secureserver.net
.sxb1.secureserver.net
$
And greping through the original blacklist I do get the same hits:
user@host:~/Downloads/blacklists$ grep -rni "secureserver.net"
webmail/domains:62:email.secureserver.net
phishing/domains:151369:ams3.secureserver.net
phishing/domains:216639:iad2.secureserver.net
phishing/domains:221982:ip.secureserver.net
phishing/domains:264752:phx3.secureserver.net
phishing/domains:288384:sin3.secureserver.net
adult/domains:242315:ams3.secureserver.net
adult/domains:1215507:iad2.secureserver.net
adult/domains:1247472:ip.secureserver.net
adult/domains:1701605:phx3.secureserver.net
adult/domains:2083560:sin3.secureserver.net
malware/domains:151574:ams3.secureserver.net
malware/domains:218268:iad2.secureserver.net
malware/domains:223617:ip.secureserver.net
malware/domains:267343:phx3.secureserver.net
malware/domains:299440:sxb1.secureserver.net
publicite/domains:2283:images-pw.secureserver.net
user@host:~/Downloads/blacklists$
I still don't understand where this match for the ip is comming from. Can someone explain it to me (and tell how to resolve the issue)?
Thanks in advance!
Simon
-
where this match for the ip is comming from
not ip. ptr record:
2020-12-05T21:50:04 squid .546 kid1| 28,3| DomainData.cc(115) match: aclMatchDomainList: 'ip-107-180-24-240.ip.secureserver.net' foundyou can see it address in blacklist also:
adult/domains:1247472:ip.secureserver.nethow to resolve the issue
try to add livingsoilssymposium.ca to whitelist at Forward Proxy -> Access Control List
-
where this match for the ip is comming from
not ip. ptr record:
Thanks for the explanation!
you can see it address in blacklist also:
adult/domains:1247472:ip.secureserver.net
This is what I don't understand yet completely. Where's the connection between the blacklist entry ip.secureserver.net and the domain livingsoilssymposium.ca or IP address 107.180.24.240? So is it like this: Squid checks the domain livingsoilssymposium.ca (no match), asks a DNS server and receives the IP address 107.180.24.240, checks the IP address (no match), does a reverse DNS lookup for the IP address and receives the domain ip.secureserver.net, checks the domain (match found!)? Did I get it right?
try to add livingsoilssymposium.ca to whitelist at Forward Proxy -> Access Control List
Tried that, but didn't work. So if the white list should overrule the blacklist, how to debug the issue if it doesn't work?
Thanks!
Simon
-
So is it like this: Squid checks the domain livingsoilssymposium.ca (no match), asks a DNS server and receives the IP address 107.180.24.240, checks the IP address (no match), does a reverse DNS lookup for the IP address and receives the domain ip.secureserver.net, checks the domain (match found!)? Did I get it right?
I don't see a request for https://livingsoilssymposium.ca in your logs.
I see https://107.180.24.240/ in error page and logs.
So squid checks ip - nothing. checks ptr - match.
OPN squid blacklists works with dstdomain directive (check domain in url and ptr if requested by ip in url)
OPN squid whitelist works with url_regex directive
So if you want to go to https://107.180.24.240/ - whitelist it
-
I got a little confused asking the question for the first time. I started a new thread (as we're now running the new OPNsense version) to understand the problem and find a sustainable solution: https://forum.opnsense.org/index.php?topic=21305.0 (https://forum.opnsense.org/index.php?topic=21305.0)
Simon