Access Denied when connecting to one site with transparent, remote-acl

[themadwizard]

Hello!  I am having a strange issue that I cannot seem to run down.

When anyone on my network tries to browse to https://idahoparcels.us they receive this error message:

Code: [Select]

The following error was encountered while trying to retrieve the URL: https://104.238.74.120/*

    Access Denied.

Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.

Your cache administrator is admin

Generated Wed, 08 Jul 2020 03:59:09 GMT by network (squid/4.11)
104.238.74.120 is the IP that the site is hosted on.  I get the same message if I try to go to the IP address directly.  This site works just fine if I check it from outside the network.  I have tried everything I can think of, including putting this on the no-bump-ssl list and on the whitelist, both by FQDN and by IP, but I get the same result every time.  The certificate returned is the internal cert, just like when any other site comes up against the ACL.  I have other sites in the whitelist and they work just fine. 

I have the proxy set to Transparent, Enable SSL Inspection, and Log SNI information only.  All other sites work correctly.

I also have tons of these errors in the log, but I don't think they are related:

Code: [Select]

SendEcho ERROR: sending to ICMPv6 packet to [2620:1ec:bdf::10]: (65) No route to host
When I look in the access log for idahoparcels.us, I get this:

Code: [Select]

2020-07-07T20:59:09.630000 0 192.168.0.110 NONE/403 3682 GET https://idahoparcels.us/favicon.ico - HIER_NONE/- text/html
2020-07-07T20:59:09.360000 0 192.168.0.110 NONE/403 3682 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:59:05.140000 0 192.168.0.110 NONE/403 3682 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:56:03.520000 0 192.168.0.110 NONE/403 3682 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:52:49.960000 0 192.168.0.110 NONE/403 3682 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:44:20.790000 0 192.168.0.110 NONE/403 3682 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:40:39.300000 0 192.168.0.110 NONE/403 3729 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:40:33.780000 95 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:40:31.150000 105 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:40:26.060000 91 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/favicon.ico - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:40:25.630000 115 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:33:45.340000 0 192.168.0.110 NONE/403 3729 GET https://idahoparcels.us/favicon.ico - HIER_NONE/- text/html
2020-07-07T20:33:45.080000 0 192.168.0.110 NONE/403 3729 GET https://idahoparcels.us/wordpress/ - HIER_NONE/- text/html
2020-07-07T20:33:31.910000 154 192.168.0.110 TCP_MISS/200 26666 GET http://idahoparcels.us/favicon.ico - ORIGINAL_DST/104.238.74.120 image/x-icon
2020-07-07T20:33:31.330000 128 192.168.0.110 TCP_MISS/200 1007 GET http://idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
If I search for  104.238.74.120, I get:

Code: [Select]

2020-07-07T21:03:14.520000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T21:03:14.340000 0 192.168.0.110 NONE/403 3682 GET https://104.238.74.120/* - HIER_NONE/- text/html
2020-07-07T21:03:14.340000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:14.370000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:13.780000 0 192.168.0.110 NONE/403 3682 GET https://104.238.74.120/* - HIER_NONE/- text/html
2020-07-07T20:59:13.760000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:09.630000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:09.570000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:09.360000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:05.370000 6 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:59:05.120000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:56:03.640000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:56:03.500000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:55:35 0 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:53:42.760000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:53:07.950000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:53:07.900000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:52:56.720000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:52:56.430000 0 192.168.0.110 NONE/403 3682 GET https://104.238.74.120/* - HIER_NONE/- text/html
2020-07-07T20:52:56.410000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:52:50.160000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:52:49.940000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:44:21 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:44:20.780000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:40:40.450000 4 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:40:39.280000 3 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:40:33.780000 95 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:40:31.150000 105 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:40:26.060000 91 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/favicon.ico - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:40:25.630000 115 192.168.0.110 TCP_MISS/404 629 GET http://www.idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
2020-07-07T20:34:09.710000 0 192.168.0.110 NONE/403 3729 GET https://104.238.74.120/favicon.ico - HIER_NONE/- text/html
2020-07-07T20:34:09.690000 5 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:09.660000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:09.290000 0 192.168.0.110 NONE/403 3729 GET https://104.238.74.120/* - HIER_NONE/- text/html
2020-07-07T20:34:09.270000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:09.170000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:08.130000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:07.090000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:06.050000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:05.010000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:03.910000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:03.740000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:03.540000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:00.220000 3 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:34:00.080000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:56.910000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:56.770000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:54.430000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:54.140000 3 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:54.050000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:45.330000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:45.290000 5 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:45.060000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:44.870000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:43.830000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:42.790000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:41.750000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:39.280000 3 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:39.090000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:38.970000 2 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:33.530000 13 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:33.150000 1 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:32.990000 175 192.168.0.110 TCP_DENIED/200 0 CONNECT 104.238.74.120:443 - HIER_NONE/- -
2020-07-07T20:33:31.910000 154 192.168.0.110 TCP_MISS/200 26666 GET http://idahoparcels.us/favicon.ico - ORIGINAL_DST/104.238.74.120 image/x-icon
2020-07-07T20:33:31.330000 128 192.168.0.110 TCP_MISS/200 1007 GET http://idahoparcels.us/ - ORIGINAL_DST/104.238.74.120 text/html
Does anyone have any ideas?

[Amr]

hello themadwizard,

This is a normal response from squid "TCP_DENIED/200" when you try to reach the site with its IP (the ip isn't in the whitelist, and squid uses the DNS name to filter).
However from the log when you tried to reach the site with its name
TCP_MISS/200: means the client was allowed to access the site (200 means OK) but the site was not cached on squid (TCP_MISS).
15 minutes later it was not found 404
then NONE/403: indicating forbidden and none Squid delivered an unusual response or no response at all
you can find more about the codes from here
https://wiki.squid-cache.org/SquidFaq/SquidLogs
to troubleshoot: first are you sure you can connect fine to the site without the proxy?
if so try to set a manual configuration to connect to the proxy (You can do so in firefox).
Finally, try resetting the cached files and certificates from support tab > Reset and restart the proxy

[themadwizard]

Unfortunately, this issue still keeps cropping up from time to time and I am unable to determine the cause.  I am able to work around it by whitelisting the IP address that the error page serves up, but that is a terrible band-aid and does nothing to indicate what the actual problem is.  Does anyone have any suggestions where in the system I would look to see why Access Control is denying these IPs?

idahoparcels.us  (104.238.74.120)
hillmeat.com (104.238.74.120)
kwausa.com (184.168.131.241)
and some GoDaddy control panel at 104.238.65.135

Access Denied occurs whether it is http or https.  None of these sites fit into any of the ACL categories, and it is the IP Address that is listed on the Access Denied page, not the website. 

I am completely baffled.

[themadwizard]

And now, it is blocking 151.101.193.21 out of nowhere, which is a Fastly IP.  This is preventing purchases via PayPal.  What the hell is going on?

[spetrillo]

Are you running Pi-Hole also?

[themadwizard]

No.  The opnsense router is providing DNS as well as proxy, content filtering, etc.

[themadwizard]

So, even more sites keep getting blocked, generally by IP.

Now, I even get the message when I browse to the opnsense firewall by the internal DNS name.  I can only get to it if I go to the IP directly.

Any ideas?

[Amr]

I used to have that problem not sure how it solved itself out, I remember that when I manually configured the client to use the proxy I didn't get that error (ofc that's is not a solution) but this is probably DNS-Squid compatibility issue (the name is resolved first then passed as IP to squid).
I made tons of changes to squid configuration and FW to pinpoint the fault, so if you can share your configuration for squid we might find something.

[Hachiman]

Hello,
i'm currently setting up my OPNsense and network and have the same issue.
I have installed a proxmox server, and I want to block internet access for it except the update repositories.

So i configured web proxy in transparent mode and HTTP only.
There I whitelisted the necessary domains.

If I try to update, then errors occur in proxmox and the web proxy log.
Setting the proxy manually in proxmox along with a FW rule didn't help and isn't a real solution.
Bypassing the proxy by allowing direct access works normally.
Is there any way to fix this?


EDIT:
I found the solution, my whitelist item was wrong.
The hints in OPNsense displayed as "full help" are kind of confusing.
To get the expressions right i recommend this tool to learn and test them:
https://www.regextester.com/