Show Posts
1
Web Proxy Filtering and Caching / Squid Proxy and local sites
« on: November 21, 2022, 07:46:17 am »
I'm trying to setup OpnSense as a web proxy/filter in a business environment and I'm so close to getting everything working. The only issue I have left is that for some reason squid cant access internal web sites. I'll describe the environment a bit better because I think I know what's happening I just don't know how to fix it.
The environment has multiple vLANs. There are a few internal vLANs and one DMZ vLAN (a small network all outbound traffic must go through before getting to the internet). OpnSense (and therefor squid) sits with its LAN interface on the primary local vLAN and its WAN interface in the DMZ. All the local vLANs are connected and can be routed between (I have them all configured and squid works on all of them), but the DMZ is isolated with no route between the DMZ and any of the other vLANs.
Squid can connect to any server on the internet just fine, and also any server in the DMZ (same interface) but it cant connect to any web server on the LAN interface. While connections to these servers shouldn't be going through the proxy, I still need to get this working as occasionally (read often) new sites get added and it can be weeks before anyone remembers to add it to the exclusions list.
The error is a very unhelpful "Read Error. The system returned: [No Error]" I get this for both HTTP and HTTPS.
I know the issue isnt a DNS issue as many of the servers in the DMZ can be accessed by an internal name (opnsense does it lookups from a server on the LAN side of the connection) and these can all be accessed using the internal name. For a while I thought it was a certificate issue (because I was getting one of those) but I've sorted that out and I still get the same error.
I suspect the issue is no routing between the DMZ (which is opnsens's gateway) and the local LANs. I've tried doing a sneaky allow rule for any connections from opnsense's WAN IP to to local networks, but I either got it wrong or it didn't help (and it cant stay that way anyway even if it fixed it, it was just for testing). I dont know where this would be logged or how to fix it (or even test, I can ping internal sites just fine).
The environment has multiple vLANs. There are a few internal vLANs and one DMZ vLAN (a small network all outbound traffic must go through before getting to the internet). OpnSense (and therefor squid) sits with its LAN interface on the primary local vLAN and its WAN interface in the DMZ. All the local vLANs are connected and can be routed between (I have them all configured and squid works on all of them), but the DMZ is isolated with no route between the DMZ and any of the other vLANs.
Squid can connect to any server on the internet just fine, and also any server in the DMZ (same interface) but it cant connect to any web server on the LAN interface. While connections to these servers shouldn't be going through the proxy, I still need to get this working as occasionally (read often) new sites get added and it can be weeks before anyone remembers to add it to the exclusions list.
The error is a very unhelpful "Read Error. The system returned: [No Error]" I get this for both HTTP and HTTPS.
I know the issue isnt a DNS issue as many of the servers in the DMZ can be accessed by an internal name (opnsense does it lookups from a server on the LAN side of the connection) and these can all be accessed using the internal name. For a while I thought it was a certificate issue (because I was getting one of those) but I've sorted that out and I still get the same error.
I suspect the issue is no routing between the DMZ (which is opnsens's gateway) and the local LANs. I've tried doing a sneaky allow rule for any connections from opnsense's WAN IP to to local networks, but I either got it wrong or it didn't help (and it cant stay that way anyway even if it fixed it, it was just for testing). I dont know where this would be logged or how to fix it (or even test, I can ping internal sites just fine).