[HAProxy] [updated] HTTPS passthrough

[tapnl]

Hi,

I am struggling to simply let HTTPS traffic to my servers pass trough HAPorxy. HTTP works fine.
For the HTTPS traffic, I have a separate public service, real servers, conditions, rules, etc setup.

I roughly have the following setup:

WAN with fixed IP -> OPNSENSE running HAPROXY -> VM running multiple docker behind Traefik.

Traefik handles all the SSL from the VM, and I am happy with that and I want to keep it that way.
I want HAProxy to pass through the HTTPS without any interference. But I am not able to figure how to do it.
Can someone point me in the right direction, because I find the documentation not very clear on this.

Thx for any pointers.

.

[tapnl]

28/04/2021

Spend quite some time on it.
I have debugged it - fixed it to the following point.

I am able to passthrough ssl/tls traffic on my static ip for a certain port (testing purposes). This is done by defining a specific backend in the frontend service. The result is that xxx.domain.com and yyy.domain.com ending up at the same server in my network serving a test website with ssl/tls enabled.

Very simple:

https://xxx.domain.com:8443 -----|
                                                   |
                                                   |----> server 1 (serving xxx.domain.com)
                                                   |
https://yyy.domain.com:8443 -----|

Obviously there is a server 2 and actually yyy.domain.com should go to server 2.

I understand that I need to do something with SNI (Server Name Indication).
This post was helpfull in understanding the concept:
https://www.cloudflare.com/en-gb/learning/ssl/what-is-sni/

But I am not able to figure out - how I can set this up in opnsense. Once again any pointers would be appreciated.

[quinniedid]

I just spend several hours trying to figure this out myself. Just in case you are still wanting to do this or someone finds this thread, this is what I have found needs to be done to get this accomplished.

You will need a static IP to do this as I have outlined, otherwise you will need to check the box next to Disable web GUI redirect rule in System -> Settings -> Administration. You will also need to change the TCP Port to something other than 443. Note you will need to use this port with HTTPS to access your firewall moving forward. It should automatically redirect you to the new port once you apply the changes.

• You need to ensure that you have setup all of your real servers, and that they are pointing to the correct internal IP and port. You will need to create a server for each port you want to route to one for HTTP and one for HTTPS, or just one if that is all you need.
• Go to Virtual Services -> Backend Pools. To simplify things, we will be using the backend to do our routing. Create a backend for both or either HTTP and HTTPS. For HTTP, create a new backend and give it a name, I used "http". In the Servers section, you will need to add all of your HTTP web servers. Leave everything else as default, and Save. For HTTPS, create a new backend and give it a name, I used "HTTPS-SNI". Change Mode to TCP (Layer4). Add all of your HTTPS web servers to the Servers section. Leave everything else as default, and Save.
• We now need to create conditions to get traffic routed correctly. Go to Rules & Checks -> Conditions. You will need to make separate rules for HTTP and HTTPS if you have both that are needed, otherwise just do the ones you need. Create a new condition. Give it a name. Condition type for HTTPS should be SNI TLS extension contains (TCP request content inspection) and for HTTP it should be Host contains. In the Parameters field, it should be the FQDN of your website (example.com or sub.example.com, etc.). Make a new condition for both or either HTTP and HTTPS for each FQDN you want to route.
• Now go to Rules & Checks -> Rules. You will need to create a rule for each condition that you had setup previously.
  
  Create a new rule and give it a name, usually the same name as your condition. In the Select conditions section, select the corresponding condition. Should only be one selected here. In the Execute function section, select Override server in Backend Pool. In the Use server section select your web server you had created earlier that you want traffic routed to. Save. Do this for each condition.
  
  For your each of your backend pools, you will need to edit it and in the Select Rules section you will need to add all the rules you had just created for all your HTTP web servers or for your HTTPS backend pool all your HTTPS web servers. Save.
• Now go to Virtual Services -> Public Services. We will need to create a new public service for both HTTP and HTTPS, or just one depending on your needs.
  
  For HTTP, create a new public service and give it a name, I used HTTP. In the Listen Addresses section you will need to put your WAN IP followed with a colon and port (1.2.3.4:80) or if you have a dynamic WAN IP you would put (0.0.0.0:80). In the Default Backend Pool section you will need to select your HTTP backend pool you created earlier. Leave everything else default. Save.
  
  For HTTPs, create a new public service and give it a name, I used HTTPS. In the Listen Addresses section you will need to put your WAN IP followed with a colon and port (1.2.3.4:443) or if you have a dynamic WAN IP you would put (0.0.0.0:443). In the Type section you need to select TCP. In the Default Backend Pool section you will need to select your HTTPS backend pool you created earlier. In the top left corner of the window, enable the slider Advanced. In the section Option pass-through put
  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }
  Leave everything else default. Save.
• Now go to Settings -> Service, and check the box Enable HAProxy. Apply.

If you haven't already setup firewall rules to all traffic in to HAProxy here is what I did.

• Go to Firewall -> Aliases. Create a new alias and name it Websrv_Ports or whatever you would like. For Type, select Port(s). In the Content section put 80 443. Hit tab after each port to add the entry. Add a description if you would like. Save.
• Go to Firewall -> Rules -> WAN (Or whatever you specified for your WAN interface). Create a new rule. Change section Protocol to TCP/UDP. Change section Destination to WAN address (Or whatever you specified for your WAN interface, followed by address). Change section Destination port range to Wensrv_Ports (Or whatever you named the alias earlier). Save.

That should be it! You should have a functioning reverse proxy that handles both HTTP and HTTPS traffic and routes it accordingly to the web servers behind your firewall.

[tapnl]

Hi,

Thanks for your post and explanation. It did not work out really for me. A one-on-one "copy" of your config is functioning, but I think it is mainly driven by assigning a dedicated backpool in step 5. As I tested it, this does not make it possible to have to HTTPS backends - as you override it.

Where I want to end is:

         managed by haproxy
{--------------------------------------}
https://www.domain1.com -----|
                                              |
https://sub1.domain1.com -----|---> server1 (running multiple dockers SSL and proxy managed by traefik)
                                              |
https://sub2.domain1.com -----|


         managed by haproxy
{--------------------------------------}
https://www.domain2.com -----|
                                              |---> server2 (running multiple dockers SSL and proxy managed by traefik)
https://sub1.domain2.com -----|

I was hoping that:

Code: [Select]

SNI TLS extension contains (TCP request content inspection)
In the conditions section combined with the addtion in the public service of:

Code: [Select]

tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }

would do the trick. Unfortunately, this did not work out.
Any ideas/pointers?

[huuich]

Hi,

Thanks for your post and explanation. It did not work out really for me. A one-on-one "copy" of your config is functioning, but I think it is mainly driven by assigning a dedicated backpool in step 5. As I tested it, this does not make it possible to have to HTTPS backends - as you override it.

Where I want to end is:

         managed by haproxy
{--------------------------------------}
https://www.domain1.com -----|
                                              |
https://sub1.domain1.com -----|---> server1 (running multiple dockers SSL and proxy managed by traefik)
                                              |
https://sub2.domain1.com -----|


         managed by haproxy
{--------------------------------------}
https://www.domain2.com -----|
                                              |---> server2 (running multiple dockers SSL and proxy managed by traefik)
https://sub1.domain2.com -----|

I was hoping that:

Code: [Select]

SNI TLS extension contains (TCP request content inspection)
In the conditions section combined with the addtion in the public service of:

Code: [Select]

tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }

would do the trick. Unfortunately, this did not work out.
Any ideas/pointers?

You can follow this guide to make it works! https://forum.opnsense.org/index.php?topic=18538.msg84958#msg84958

[Syon]

I have an additional question...
For me it is not possible to enable the send-proxy Option.
(Advanced in backend --> Proxy Protocol V 1or2)
The option seems to be unused...
My Browser shows only "PR_END_OF_FILE_ERROR"

Thx in advanced

[Syon]

Oh... sorry... I was to blind..
Solution: Edit Server --> advanced mode --> Option pass-through "send-proxy".



   

[huuich]

Oh... sorry... I was to blind..
Solution: Edit Server --> advanced mode --> Option pass-through "send-proxy".

Did you use HTTPS passthrough ok?

[Syon]

Sorry for the delay... I pass everything...

Code: [Select]

# Frontend: http ()
frontend http
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: xxx_ACL_http
acl acl_625d3134619400.72128715 hdr_sub(host) -i xxx.xxxx.de

use_backend xxx_http_proxy-protocol_backend if acl acl_625d3134619400.72128715

# Frontend: https ()
frontend https
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp
    # tuning options
    timeout client 30s

    # logging options
    # ACL: xxx_ACL_https
    acl acl_625d3134619400.72128715 req.ssl_sni -m sub -i xx.xxxx.de

    # ACTION: xxx_https_rule
    use_backend xxx_https_proxy-protocol_backend if acl_625d3134619400.72128715

# Backend: xxx_http_proxy-protocol_backend ()
backend xxx_http_proxy-protocol_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server xxx_http IP:80 send-proxy-v2 check-send-proxy send-proxy

# Backend: xxx_https_proxy-protocol_backend ()
backend xxx_https_proxy-protocol_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server xxx_https IP:443 send-proxy check-send-proxy send-proxy