Messages

Oh... sorry... I was to blind..
Solution: Edit Server --> advanced mode --> Option pass-through "send-proxy".

Did you use HTTPS passthrough ok?

@cookiemonster Thank for your comment about back to original purpose, I'll create a new topic relate my questions. Thanks.

@Bunch: I'm very thankful to you for your help. Best Regards!

The modification method is mentioned in Nginx guide

Should be modified as below (haven't tested it)

Your code "haven't tested but working for me with http website, but when I change to https with certbot

Code Select Expand

certbot --nginx -d c.mywebsite.com

My https website can not access though I've change my server port to 443, this is my nginx website config

Code Select Expand

server {
listen 32401 proxy_protocol;
listen [::]:32401 proxy_protocol;

listen [::]:443 ssl ipv6only=on proxy_protocol; # managed by Certbot
    listen 443 ssl proxy_protocol; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/c.mywebsite.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/c.mywebsite.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

server_name c.mywebsite.com;

root /var/www/html;

location / {
try_files $uri $uri/ /yourls-loader.php$is_args$args;
proxy_set_header X-Real-IP $proxy_protocol_addr;
proxy_set_header X-Forwarded-For $proxy_protocol_addr;
}

location ~ \.php$ {
include fastcgi.conf;

fastcgi_index index.php;

fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
}
}

and my current HAProxy configure

Code Select Expand

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    6
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log /var/run/log local0 info

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend (DISABLED): https_passthrough ()

# Frontend: tcp_front (Listen to 0.0.0.0:443, TCP SNI handler, redirect if v2ray))
frontend tcp_front
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp
    default_backend SSL_back
    # tuning options
    timeout client 30s

    # logging options
    # ACL: traffic_ssl
    acl acl_601a842f14cee3.17646593 req_ssl_hello_type 1

    # ACTION: tcp_request_content_accept_ssl
    tcp-request content accept if acl_601a842f14cee3.17646593
    # ACTION: tcp_request_inspect_delay
    # NOTE: actions with no ACLs/conditions will always match
    tcp-request inspect-delay 5s
    # ACTION: hmdir_ru
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/622eeaa3044ba7.74145133.txt)]

# Frontend: http_front (Listening on 192.168.64.1:80)
frontend http_front
    bind 192.168.64.1:80 name 192.168.64.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: NoSSL_condition
    acl acl_622eebaf197419.36314953 req.ssl_ver gt 0

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_622eebaf197419.36314953

# Frontend: ssl_front (Listening on 192.168.64.1:443)
frontend ssl_front
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 192.168.64.1:443 name 192.168.64.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/622eef9a9d7268.16491040.certlist
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 15m

    # logging options

    # ACTION: hmdir_ru
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/622eeaa3044ba7.74145133.txt)]

# Backend: hmdir_back ()
backend hmdir_back
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server hmdir 192.168.82.11:443 ssl verify none send-proxy-v2 check-send-proxy

# Backend (DISABLED): PLEX_backend ()

# Backend (DISABLED): acme_challenge_backend (Added by Let's Encrypt plugin)

# Backend: SSL_back ()
backend SSL_back
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server SSL_server 192.168.64.1 send-proxy-v2 check-send-proxy

# Backend (DISABLED): ctl_backend ()

Could you take a look and guide me how to run https on my website?

For the log config, I forgot rather it is /etc/nginx/nginx.conf by default
You need to change the log format inside http{} to something like

Code Select Expand

log_format combined '$proxy_protocol_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';

Fire I use above code has a error, so I google and found a solution for working log, I post here for anyone need

Code Select Expand

log_format my_log '$proxy_protocol_addr - $remote_user [$time_local] '
                       '"$request" $status $body_bytes_sent '
                       '"$http_referer" "$http_user_agent"';

access_log /var/log/nginx/access.log my_log;

For testing why it won't work in your case,
You can try not using Proxy Protocol first. If you can access your backend, that means your webserver config is not accepting proxy protocol (It won't work both way at the same time)
You need to use Proxy Protocol and modify webserver config ( for example, /etc/nginx/sites-enabled/default) at the same time

Similar test case mentioned in previous post

Example: my v2ray server is living in TCP mode (although I don't care about the IP log)
I will get router IP in log if I don't user proxy protocol.
I cannot reach v2ray server if I use proxy protocol in haproxy but not changing any config of v2ray server (probably rejected by the server itself)
I will get correct client IP if I use proxy protocol in haproxy and changing the config of v2ray to accept proxy protocol

My /etc/nginx/sites-enabled is

Code Select Expand

server {
listen 32401;
listen [::]:32401;

server_name c.mywebsite.com;

root /var/www/html;

location / {
try_files $uri $uri/ /yourls-loader.php$is_args$args;
}

location ~ \.php$ {
include fastcgi.conf;

fastcgi_index index.php;

fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
}


How could use Proxy Protocol and modify webserver config with my HAProxy config above?
Sorry for basic question. Thank you for detailed explaination.

@Bunch Thank you for detailed guide and attached config. I've tried to simulate like your server but now my server not working

This is my config

Code Select Expand

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    6
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log /var/run/log local0 info

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend (DISABLED): https_passthrough ()

# Frontend: tcp_front (Listen to 0.0.0.0:443, TCP SNI handler, redirect if v2ray))
frontend tcp_front
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp
    default_backend SSL_back
    # tuning options
    timeout client 30s

    # logging options
    # ACL: traffic_ssl
    acl acl_601a842f14cee3.17646593 req_ssl_hello_type 1

    # ACTION: tcp_request_content_accept_ssl
    tcp-request content accept if acl_601a842f14cee3.17646593
    # ACTION: tcp_request_inspect_delay
    # NOTE: actions with no ACLs/conditions will always match
    tcp-request inspect-delay 5s
    # ACTION: hmdir_ru
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/622eeaa3044ba7.74145133.txt)]

# Frontend: http_front (Listening on 192.168.64.1:80)
frontend http_front
    bind 192.168.64.1:80 name 192.168.64.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: NoSSL_condition
    acl acl_622eebaf197419.36314953 req.ssl_ver gt 0

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_622eebaf197419.36314953

# Frontend: ssl_front (Listening on 192.168.64.1:443)
frontend ssl_front
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 192.168.64.1:443 name 192.168.64.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/622eef9a9d7268.16491040.certlist
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 15m

    # logging options

    # ACTION: hmdir_ru
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/622eeaa3044ba7.74145133.txt)]

# Backend: hmdir_back ()
backend hmdir_back
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server hmdir 192.168.82.11:32401 send-proxy-v2 check-send-proxy

# Backend (DISABLED): PLEX_backend ()

# Backend (DISABLED): acme_challenge_backend (Added by Let's Encrypt plugin)

# Backend: SSL_back ()
backend SSL_back
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server SSL_server 192.168.64.1 send-proxy-v2 check-send-proxy

# Backend (DISABLED): ctl_backend ()

My hmdir_ru is link to PUBLIC_SUBDOMAINS_map map file with content:

Code Select Expand

# public access subdomains
c hmdir_back

Could you share content of map file in local_map_ru in this config

Code Select Expand

# ACTION: local_map_ru
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/623230d7bffd04.94702836.txt)] if acl_62123a1cebe813.09309501

And ACL http in this config

Code Select Expand

# logging options
    # ACL: http
    acl acl_62123bbee27260.60165685 ssl_fc

Thank you!

If all SSL certificate are handled by webserver themselves.
Follow #176, step 1-4 will allow 0_SNI_FRONTEND to recognize TLS package and redirect under TCP mode

Note: All backend redirect from 0_SNI_FRONTEND should be in TCP mode
HAProxy only work with server that using TLS package. For those server that doesn't contain TLS package (for example, game servers), HAProxy won't work. Although you can set default backend server to game server, but there's only one default backend server.

BTW, for backend server getting source IP, enabling X-Forwarded-For header for all frontend should work
However, this one only apply to HTTP.
reference
reference 2

I've follow your guide but though my backend server ctl_backend has code

Code Select Expand

http-request set-header X-Forwarded-Proto https if { ssl_fc }
http-request set-header X-Real-IP %[src]

But it can not get source ip, here is my full config, could you show me how to get source ip? Or I have to have 2 OPNSense, one for SSL passthrough and one use Let's Encrypt Wildcard Certificates like this topic guide? Could 1  OPNSense sever run both nginx reverse proxy and HAProxy?

Code Select Expand

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    6
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log /var/run/log local0 info

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend (DISABLED): https_passthrough ()

# Frontend: 0_SNI_frontend (Listening on 0.0.0.0:80, 0.0.0.0:443)
frontend 0_SNI_frontend
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp
    # tuning options
    timeout client 30s

    # logging options
    # ACL: traffic_ssl
    acl acl_601a842f14cee3.17646593 req_ssl_hello_type 1

    # ACTION: request_content_accept_ssl
    tcp-request content accept if acl_601a842f14cee3.17646593
    # ACTION: request_inspect_delay
    # NOTE: actions with no ACLs/conditions will always match
    tcp-request inspect-delay 5s
    # ACTION: PUBLIC_SUBDOMAINS_map-rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/622eeaa3044ba7.74145133.txt)]

# Frontend (DISABLED): 1_HTTP_frontend (Listening on 192.168.64.1:80)

# Frontend (DISABLED): 1_HTTPS_frontend (Listening on 192.168.64.1:443)

# Backend: PLEX_backend ()
backend PLEX_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    http-request set-header X-Real-IP %[src]
    http-reuse safe
    server PLEX_server 192.168.82.11:32400

# Backend (DISABLED): acme_challenge_backend (Added by Let's Encrypt plugin)

# Backend (DISABLED): SSL_backend ()

# Backend: ctl_backend ()
backend ctl_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    http-request set-header X-Real-IP %[src]
    http-reuse safe
    server ctl_server 192.168.82.11:32401

You have to set your backends and frontends to HTTP Mode.
Also disable SSL offloading on the frontends.

But I can't guarantee for sure that it will work.
TCP Mode will never (with a few exceptions) work because there is no header in the packets that would tell HAProxy which service to send the traffic to.

HTTP Mode could work, but you might need to create some "http header contains..." conditions.

I can do that using SSL passthrough follow step by step this guide from @alh

Every works ok except my Nginx log can not receive real ip from client, it's only show ip  local of my OPNSense server in file /var/log/nginx/access.log

Do you think can mix SSL passthrough with your guide (edit 0_SNI_frontend with rules from SSL passthrough and Default Backend Pool) or any solution to receive real ip from client with SSL passthrough? Thanks!

Code Select Expand

This should work for any TCP-based SSL/TLS encrypted service in passthrough (HAProxy: TCP) mode... It does NOT work for STARTTLS!

In this example I use TCP port 443.

HAProxy plugin: Create "Real Server" (enter name, IP/FQDN and port number if different from 443, the rest can be left at default)
HAProxy plugin: Create "Backend Pool" (enter name, set mode to TCP and select the real server from step 1)
HAProxy plugin: Create "Condition" (enter name ["traffic_ssl"], condition type is "custom condition (option pass-through)" with value "req_ssl_hello_type 1")
HAProxy plugin: Create "condition" (enter name ["myservice_sni"], condition type is "SNI TLS extension matches (TCP request content inspection)" with value "myservice.example.com" or whatever your FQDN is)
HAProxy plugin: Create "Rule" (enter name ["request_inspect_delay"], select no condition, function is "tcp-request inspect delay" with value "5s" or whatever suits you)
HAProxy plugin: Create "Rule" (enter name ["request_content_accept_ssl"], select condition of 3 ["traffic_ssl"], function is "tcp-request content accept")
HAProxy plugin: Create "Rule" (enter name ["myservice_sni"], select condition of 4 ["myservice_sni"], function is "Use specific backend pool" with your pool from 2)
HAProxy plugin: Create "Public service" (enter name ["https_passthrough"], choose a listen address [":443" for all], type is "TCP" and select the 3 rules created earlier)
HAProxy plugin: Enable plugin or test/apply
Firewall: allow incoming traffic to WAN (address) or whatever for TCP port 443.

That works at least for me. If you have double NAT you would need to disable port randomization for the proxied port...

Does that help you?

I want to set up HAProxy just for routing traffic based on URLs (https://xyz.domain.com goes to server 1 and https://abc.domain.com goes to server 2, etc...).
All SSL stuff for the destination web servers is being handled by a separate Linux certificate server and the web servers themselfes, independent from OPNsense/HAProxy. HAProxy is really only needed for routing traffic based on URLs, nothing more, nothing less.I want to set up HAProxy just for routing traffic based on URLs (https://xyz.domain.com goes to server 1 and https://abc.domain.com goes to server 2, etc...).
All SSL stuff for the destination web servers is being handled by a separate Linux certificate server and the web servers themselfes, independent from OPNsense/HAProxy. HAProxy is really only needed for routing traffic based on URLs, nothing more, nothing less.

Did you find solution for your purpose? I wanna do same like you.

[First.]

First.
The entry "plex PLEX_backend" in the mapfile means that you will have to access it using the "plex" subdomain. --> In your case "plex.mywebsite.com"!

Alternatively just set the PLEX_backend as default backend on your HTTPS_frontend.

Second.
http will always get redirected to https. This is intended and you will most probably want this! This is configured using the HTTPtoHTTPS_rule and NoSSL_condition.

Third.
Apart from the above your config looks good. (just took a very short look at it)

Fourth.
You might have to disable the SSL checkbox in the PLEX_server settings.
But only if you are REALLY accessing it locally using http://IP:32400 and the service is NOT redirecting you to HTTPS. But I highly doubt this since Plex is running on a self-signed SSL cert by default...

You are right, mysubdomain.mywebsite.com, and follow your guide I've edit my Map File to "mysubdomain PLEX_backend" and "disable the SSL checkbox in the PLEX_server settings" and voila my subdomain website run https:// ok with A+ score, Thank you so much for quick and detailed reply!

P/s: I've follow your guide with cloudflare instead. (almost every settings are the same like your guide)

Thanks for detailed instructions, I've follow step by step to make a web hosting running nginx with https support.

My HAProxy Config:

Code Select Expand

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    6
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log /var/run/log local0 info

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend (DISABLED): https_passthrough ()

# Frontend: 0_SNI_frontend (Listening on 0.0.0.0:80, 0.0.0.0:443)
frontend 0_SNI_frontend
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp
    default_backend SSL_backend
    # tuning options
    timeout client 30s

    # logging options

# Frontend: 1_HTTP_frontend (Listening on 192.168.64.1:80)
frontend 1_HTTP_frontend
    bind 192.168.64.1:80 name 192.168.64.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: NoSSL_condition
    acl acl_622eebaf197419.36314953 req.ssl_ver gt 0

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_622eebaf197419.36314953

# Frontend: 1_HTTPS_frontend (Listening on 192.168.64.1:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 192.168.64.1:443 name 192.168.64.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/622eef9a9d7268.16491040.certlist
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 15m

    # logging options

    # ACTION: PUBLIC_SUBDOMAINS_map-rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/622eeaa3044ba7.74145133.txt)]

# Backend: backend_pool_abc ()
backend backend_pool_meet_huuich_vn
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s

# Backend: PLEX_backend ()
backend PLEX_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server PLEX_server 192.168.82.11:32400 ssl verify none

# Backend: acme_challenge_backend (Added by Let's Encrypt plugin)
backend acme_challenge_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server acme_challenge_host 127.0.0.1:43580

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server SSL_server 192.168.64.1 send-proxy-v2 check-send-proxy

My Map File content:

Code Select Expand

# public access subdomains
plex PLEX_backend

My nginx website config

Code Select Expand

server {
        listen 32400;
        listen [::]:32400;

        root /var/www/mywebsite.com/html;
        index index.html index.htm index.nginx-debian.html;

        server_name mywebsite.com;

        location / {
                try_files $uri $uri/ =404;
        }
}

I can go my website on internal http://192.168.82.11:32400 is ok but when I access http://mywebsite.com browser go to https://mywebsite.com and show error

Code Select Expand

503 Service Unavailable
No server is available to handle this request.

How can I fix this and show my website https://mywebsite.com ok? Thanks!

Hello,

Is it possible in the GUI to set the send-proxy flag. I need this to have the real IP address on the postfix server.

Code Select Expand

backend test
  server test.smtp 1.2.3.4:25 send-proxy

Thanks Metz

Did you find solution for your question?

I've install HAProxy using SSL passthrough follow your guide and every works ok except my Nginx log can not receive real ip from client, it's only show ip  local of my OPNSense server in file /var/log/nginx/access.log

I follow this guide from BeNe but nothing change, Do you have any solution in this? Thanks!

Code Select Expand

Services -> HAProxy -> Virtual Services -> Backend Pool -> Edit Server
Enable "advanced mode"

Field:
Option pass-through

Code: [Select]
# add X-FORWARDED-FOR
option forwardfor

# add X-CLIENT-IP
# http-request add-header X-CLIENT-IP %[src]

This should work for any TCP-based SSL/TLS encrypted service in passthrough (HAProxy: TCP) mode... It does NOT work for STARTTLS!

In this example I use TCP port 443.

• HAProxy plugin: Create "Real Server" (enter name, IP/FQDN and port number if different from 443, the rest can be left at default)
• HAProxy plugin: Create "Backend Pool" (enter name, set mode to TCP and select the real server from step 1)
• HAProxy plugin: Create "Condition" (enter name ["traffic_ssl"], condition type is "custom condition (option pass-through)" with value "req_ssl_hello_type 1")
• HAProxy plugin: Create "condition" (enter name ["myservice_sni"], condition type is "SNI TLS extension matches (TCP request content inspection)" with value "myservice.example.com" or whatever your FQDN is)
• HAProxy plugin: Create "Rule" (enter name ["request_inspect_delay"], select no condition, function is "tcp-request inspect delay" with value "5s" or whatever suits you)
• HAProxy plugin: Create "Rule" (enter name ["request_content_accept_ssl"], select condition of 3 ["traffic_ssl"], function is "tcp-request content accept")
• HAProxy plugin: Create "Rule" (enter name ["myservice_sni"], select condition of 4 ["myservice_sni"], function is "Use specific backend pool" with your pool from 2)
• HAProxy plugin: Create "Public service" (enter name ["https_passthrough"], choose a listen address [":443" for all], type is "TCP" and select the 3 rules created earlier)
• HAProxy plugin: Enable plugin or test/apply
• Firewall: allow incoming traffic to WAN (address) or whatever for TCP port 443.

That works at least for me. If you have double NAT you would need to disable port randomization for the proxied port...

Does that help you?

Also, i've read a lot of reverse-proxy guides that state the need to use x-forwarded-for option.
Is this enabled by default or is it a hidden option? How can i double check this feature is enabled in HAproxy?

Yes, there was an Option to enable/disable in the past  ::)
You can also set it on your own:

Services -> HAProxy -> Virtual Services -> Backend Pool -> Edit Server
Enable "advanced mode"

Field:
Option pass-through

Code Select Expand

# add X-FORWARDED-FOR
option forwardfor

# add X-CLIENT-IP
# http-request add-header X-CLIENT-IP %[src]

I've install HAProxy using SSL passthrough follow this guide from @alh

Every works ok except my Nginx log can not receive real ip from client, it's only show ip  local of my OPNSense server in file /var/log/nginx/access.log

I follow your guide but nothing change, Do you have any solution in this? Thanks!

Code Select Expand

This should work for any TCP-based SSL/TLS encrypted service in passthrough (HAProxy: TCP) mode... It does NOT work for STARTTLS!

In this example I use TCP port 443.

HAProxy plugin: Create "Real Server" (enter name, IP/FQDN and port number if different from 443, the rest can be left at default)
HAProxy plugin: Create "Backend Pool" (enter name, set mode to TCP and select the real server from step 1)
HAProxy plugin: Create "Condition" (enter name ["traffic_ssl"], condition type is "custom condition (option pass-through)" with value "req_ssl_hello_type 1")
HAProxy plugin: Create "condition" (enter name ["myservice_sni"], condition type is "SNI TLS extension matches (TCP request content inspection)" with value "myservice.example.com" or whatever your FQDN is)
HAProxy plugin: Create "Rule" (enter name ["request_inspect_delay"], select no condition, function is "tcp-request inspect delay" with value "5s" or whatever suits you)
HAProxy plugin: Create "Rule" (enter name ["request_content_accept_ssl"], select condition of 3 ["traffic_ssl"], function is "tcp-request content accept")
HAProxy plugin: Create "Rule" (enter name ["myservice_sni"], select condition of 4 ["myservice_sni"], function is "Use specific backend pool" with your pool from 2)
HAProxy plugin: Create "Public service" (enter name ["https_passthrough"], choose a listen address [":443" for all], type is "TCP" and select the 3 rules created earlier)
HAProxy plugin: Enable plugin or test/apply
Firewall: allow incoming traffic to WAN (address) or whatever for TCP port 443.

That works at least for me. If you have double NAT you would need to disable port randomization for the proxied port...

Does that help you?

Hi,

Thanks for your post and explanation. It did not work out really for me. A one-on-one "copy" of your config is functioning, but I think it is mainly driven by assigning a dedicated backpool in step 5. As I tested it, this does not make it possible to have to HTTPS backends - as you override it.

Where I want to end is:

         managed by haproxy
{--------------------------------------}
https://www.domain1.com -----|
                                              |
https://sub1.domain1.com -----|---> server1 (running multiple dockers SSL and proxy managed by traefik)
                                              |
https://sub2.domain1.com -----|


         managed by haproxy
{--------------------------------------}
https://www.domain2.com -----|
                                              |---> server2 (running multiple dockers SSL and proxy managed by traefik)
https://sub1.domain2.com -----|

I was hoping that:

Code Select Expand

SNI TLS extension contains (TCP request content inspection)

In the conditions section combined with the addtion in the public service of:

Code Select Expand


tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }


would do the trick. Unfortunately, this did not work out.
Any ideas/pointers?

You can follow this guide to make it works! https://forum.opnsense.org/index.php?topic=18538.msg84958#msg84958

Thanks for your reply, so what can I do to fix this?