Transparent HTTPS Proxy. Pros and cons

Started by WhiteTiger, November 15, 2023, 01:29:25 AM

Previous topic - Next topic
Sorry for the perhaps trivial question, but what are the Pros and Cons of enabling the HTTPS Transparent Proxy?

I have read the considerations regarding the dangers of activating "Man in the Middle", but I also see the difficulties in having to manage PC and smartphone configurations if you do not have direct control of them, such as in the case of corporate guests.

I wonder if by having a domain certificate (whether provided by the ISP or by Let's Encrypt) I can avoid having to configure PCs and Smartphones (which I am obliged to do if the certificate is generated internally).
However, they would still need to be configured to provide the wpad.dat and not all people are capable of doing this, nor can I do it for all those arriving from outside.

So, the question is precisely the initial one.
What Pros and Cons?
Followed closely by "What alternatives should we foresee instead?"

Thanks in advance for the clarifications.

November 15, 2023, 10:45:44 AM #1 Last Edit: November 15, 2023, 10:50:10 AM by meyergru
You cannot use a domain certificate, you need a certificate authority (CA), which is neccessarily self-signed and has to be trusted (= imported) by the clients. This is because you have to fake the target certificate on-the-fly. So no available shortcuts there.

Trust in your own CA is problematic in the case of corporate guests. Many of them would or even may not trust your CA and would not install it if instructed to do so. In all corporate environments I have seen, there are only "free WLANs" for corporate guests that are separated from the company networks. If you fear mis-use, you can route the traffic for those networks over a VPN, such that the outgoing traffic will not be attributed to you.

For you internal needs, you can use HTTPS transparent proxies, with a few exceptions. Usually, there are no security implications, IFF you keep your CA secure. Mind you, its private key has to be on the proxy/firewall, so make sure that is secure.

The exceptions are certificates that are "pinned". Some banks do that just to ensure that SSL traffic does not get terminated in the middle. Technically, your proxy plays the target server, builds a fake certificate via its own CA (which is trusted by the client, so it thinks it is connected to the correct site). Your proxy then has access to the unencrypted traffic, thus is able to analyse it. There is a second SSL connection between the proxy and the real target.

"Pinned" certificates dictate a certain hash value, so a client can see that an incorrect (fake) certificate is being used and give out a warning. For this reason, most HTTPS transparent proxies have exception lists for domains that must not be terminated at the proxy. Sometimes, they contains non-bank but trustworthy sites like Amazon.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Quote from: meyergru on November 15, 2023, 10:45:44 AM
Usually, there are no security implications, IFF you keep your CA secure. Mind you, its private key has to be on the proxy/firewall, so make sure that is secure.
That can be mitigated by using the private root CA to sign an intermediate CA once, then literally lock away the root CA private key in a safe. Use the intermediate CA for certificate signing.
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)

November 15, 2023, 11:04:02 AM #3 Last Edit: November 15, 2023, 11:06:18 AM by meyergru
Quote from: Patrick M. Hausen on November 15, 2023, 10:56:14 AM
Quote from: meyergru on November 15, 2023, 10:45:44 AM
Usually, there are no security implications, IFF you keep your CA secure. Mind you, its private key has to be on the proxy/firewall, so make sure that is secure.
That can be mitigated by using the private root CA to sign an intermediate CA once, then literally lock away the root CA private key in a safe. Use the intermediate CA for certificate signing.

Right. I meant the signing "proxy" CA, which can (and probably should from a best practice standpoint) be separate from the root CA. But since you probably do not want to handle CRLs, the signing CA, if stolen, can do damage anyway during its lifetime. You can limit that as well, but in that case, you have to replace it more often.

Even if you do employ CRLs, you would have to notice a theft first.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Quote from: meyergru on November 15, 2023, 10:45:44 AM
In all corporate environments I have seen, there are only "free WLANs" for corporate guests that are separated from the company networks. If you fear mis-use, you can route the traffic for those networks over a VPN, such that the outgoing traffic will not be attributed to you.

Can you explain better what you are suggesting?
I'm already thinking about two WLANs to balance the traffic.
Suggest using one for "Guests" traffic as well.
Basically some rule that routes all traffic from a guest VLAN to this specific WLAN?
Or have a third one only for guests?

I didn't understand what I should do with the VPN.
Buy an external VPN and then to route all guest traffic to it? Or enable one on the firewall?

After I posted my question yesterday I read this other post.

At this point I also ask myself whether it still makes sense to configure Squid in maximum detail and then end up with a "halved" engine.

As I said initially, I am testing a solution for a small company and it makes little sense to invest time and resources on an engine if there is no prospect of its future stability.
You might as well keep only the HTTP Transparent Proxy and give up the filtering, ICAP and HTTPS services.
Meanwhile looking for alternatives.

When I go into one of my employers' offices, I have the same so can explain if meyergru doesn't mind.
These offices have Wifi, multiple SSIDs. One is say corpNET that people can use to connect to and needs a particular type of central auth. Corporate laptops and corporate mobile phones connect to that one, and can access corporate resources.
There is also let's say guestNET that guests can connect to. Those can be for example for my personal work phone. That will NOT have access to corporate resources. Credentials are different too but doesn't have to be. Why segreagating SSDs and networks? Because they can't apply endpoint policies to my personal phone, so no guarantees what nasties are coming for the ride on my personal phone. Out to the internet, they're not bothered.
That's what this bit refers to (I imagine):
Quote from: meyergru on November 15, 2023, 10:45:44 AM
In all corporate environments I have seen, there are only "free WLANs" for corporate guests that are separated from the company networks. If you fear mis-use, you can route the traffic for those networks over a VPN, such that the outgoing traffic will not be attributed to you.
It's two perspectives. From the guestNET on my mobile, I will use a VPN on the personal phone if connected to it, simply because I don't want to have my traffic being potentially eavesdropped by a corporate MiM.
From the corporate perspective, if you were the admin and responsible for that traffic, thats when you could use a VPN for that traffic. All this done properly for more than a few dozens of users, starts requiring to think about commercial products.

As I was saying, I'm now doing some tests in the laboratory before implementing them definitively.
My idea was to mount Ubiquiti Access Points that allow me to create different SSIDs by immediately assigning them to different VLANs.
I then thought about creating a Captive Portal where users belonged to different groups:

  • CompanyNet, which access the company network based on firewall rules.
  • Mobile-Net, for company tablets and smartphones that can also access some services.
  • Guest-Net1 which can go to the internet, but also access limited services, such as printers.
  • Guest-Net2, who can only go to the Internet.
Users would log in with FreeRadius.

As I asked before, are you suggesting a specific WLAN dedicated to guests?

As for the VPN, I'm interested in understanding what type of VPN to implement at a corporate level for guests.
I have always thought of either a VPN that connects two remote offices or a VPN to connect the PCs of individual remote users to the main office.
Here you tell me about a VPN that starts from the main office and goes to the Internet and I haven't understood how to implement it.

November 15, 2023, 06:53:13 PM #8 Last Edit: November 15, 2023, 07:07:33 PM by meyergru
First off, it is always a good idea to segment your network into several VLANs / LANs or WLANs.

I decided to segregate them based on what they should be allowed to do. Currently, I have these zones:


  • LAN for my internal Clients and servers

  • IoT for my devices that I do not fully trust (because they could "phone home" and pierce my firewall

  • DMZ for any VMs or device that are in my network but belong to others (like a backup server)

  • Guest (kinda self-explanatory)

  • MGMT for device management (switches, APs, firewall, NAS)

All of them have a 1:1 mapping between VLANs and corresponding WLANs. Since the "zones" are defined by what they can do, why should this be different in ethernet and WLAN networks?

I also use 802.1x to protect my physical ports. "LAN -> IoT" and "MGMT -> all" traffic is allowed, anything else is forbidden (apart from specific holes).

My Guest network has internet access - I use a simple WLAN password protection, but that could be changed to time-based tickets with my Unifi APs by using the Guest portal on my Unifi Controller (which is a VM on Proxmox). Also, the Guest network has full unhindered internet access (i.e. no transparent HTTPS proxy). If I was to mistrust my Guests, I could use VPN provider and route all of the Guest network traffic over that provider. If anybody misuses this, the traffic does not originate from my "official" IPs.

I have no children, so I do not need a transparent proxy for my LAN and I do not use any fancy protection stuff like zenarmor, whitelists - but I could do that.

BTW: obviously, transparent proxies would probably not work for IoT devices, either.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Quote from: meyergru on November 15, 2023, 06:53:13 PM
... I could use VPN provider and route all of the Guest network traffic over that provider.

How do you activate the provider's VPN? I didn't understand this.

You configure a VPN to one of the common VPN providers via any suitable VPN type (Wireguard, IPsec or OPenVPN) and then you configure a route or a NAT rule for just the Guest subnet you want to go through that VPN's gateway. That way, all of your other networks use the default gateway, only the Guest subnet will be behind the VPN.

There is (quite old) writeup about this, which should work with a few adaptions. Maybe you can find more current howtos.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Quote from: meyergru on November 18, 2023, 11:35:04 AM
You configure a VPN to one of the common VPN providers via any suitable VPN type (Wireguard, IPsec or OPenVPN) and then you configure a route or a NAT rule for just the Guest subnet you want to go through that VPN's gateway. That way, all of your other networks use the default gateway, only the Guest subnet will be behind the VPN.

It's a solution I hadn't considered. I look at your report with interest.
I only ask you for a first thing; Do I need a dedicated WAN? I ask because obviously the current WAN is already used for authorized traffic.

Do you have any VPN suggestions that I can use for testing? I've never felt the need for it and I don't know about it.
A free VPN, possibly.

No, you do not need a dedicated WAN for this. The connection to the VPN provider can go through the usual WAN.

You can try ProtonVPN, there is a Free version. Obviously, the number of servers are limited with Free, such that the ping time may not be optimal and there may be less speed than with the paid version, but you don't owe anything special to your guest users. Plus they support Wireguard, which has low overhead.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Quote from: meyergru on November 15, 2023, 06:53:13 PM
...
Also, the Guest network has full unhindered internet access (i.e. no transparent HTTPS proxy). If I was to mistrust my Guests, I could use VPN provider and route all of the Guest network traffic over that provider. If anybody misuses this, the traffic does not originate from my "official" IPs.
...

Hello and happy new year.
I'm continuing my testing where we left off and following your suggestion to use a VPN for the home and guest VLAN.
I need some advice from you.
1) Do all the hosts on your VLAN use the VPN? I would like to differentiate the hosts without creating additional VPNs. I'm finding my way around using subnets. Subnets A and B use VPN, subnets C and D use WAN.
2) In your opinion, what types of devices should not pass through the VPN? Can printers, SmartTVs and IoT only pass through the WAN?
3) How can I understand from OPNsense if a device/PC is using the VPN? I use "ProtonVPN Free" and have no indication in its dashboard that the VPN is being used at that moment. In OPNsense I see traffic in the graph, but the data is generic. I would like something related to the specific host "XYZ".

Quote from: WhiteTiger on January 21, 2024, 11:55:14 PM
1) Do all the hosts on your VLAN use the VPN? I would like to differentiate the hosts without creating additional VPNs. I'm finding my way around using subnets. Subnets A and B use VPN, subnets C and D use WAN.

I neither have untrustworthy guest nor an open WLAN, so I do not use this at all. If you want to use it, what is the difference between (V)LANs and subnets? If your subnets share the same (V)LAN, the assignment of a device to a subnet is arbitrary and secure nothing.


Quote from: WhiteTiger on January 21, 2024, 11:55:14 PM
2) In your opinion, what types of devices should not pass through the VPN? Can printers, SmartTVs and IoT only pass through the WAN?

That depends on what you want to achieve: I would differentiate between devices I do not want to give away my public IP and devices that may do just that. Untrustworthy clients are of the former kind because they may take illegal actions that I do not want to have tracked. SmartTVs and printers phone home, but if you can live with that their manufacturers know your real IP, then O.K.

BTW: The primary reason for SmartTVs to reside behind a VPN IP is to circumvent regional restrictions.

Quote from: WhiteTiger on January 21, 2024, 11:55:14 PM
3) How can I understand from OPNsense if a device/PC is using the VPN? I use "ProtonVPN Free" and have no indication in its dashboard that the VPN is being used at that moment. In OPNsense I see traffic in the graph, but the data is generic. I would like something related to the specific host "XYZ".

You can see it in the firewall logs if you enable logging for the specific outgoing firewall rule - you need one anyway because you have to specify the gateway (to go over the VPN).
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+