Tutorial 2024/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

[TheHellSite]

I get that 21116 is UDP and maybe will be not possible to reverse proxy this but i need to ask is there a way to get it working ?

No, it is not possible to reverse proxy UDP traffic based on the access URL or FQDN. Unlike TCP or HTTP, which use a header with the destination hostname or URL, UDP packets do not contain any information about the intended destination.

UDP is a connectionless protocol, which means that packets are sent from the client to the server without any prior communication or setup. Each packet is independent and can be routed independently. This makes it impossible to perform URL-based routing or reverse proxying for UDP traffic.

HAProxy can still perform load balancing for UDP traffic based on IP address and port, but it cannot route based on the content of the UDP packet or the access URL or FQDN.

Because of that, if you don't run multiple RustDesk servers and therefore have no need of actually load balancing it, you can just use a simple UDP port forward to your rust desk server. You won't have any benefit with setting this up in HAProxy.

Another question is there a way to make range of port like 21115-21117 because adding some services may make the UI little overcrowded in time

No, because each server has to be unique since a reverse proxy is also a load balancer.
You can however combine them in the same backend, but only IF they server the exact same content. Otherwise packets will end up on the wrong port.

I think you actually can, however the OPNsense HAProxy GUI doesn't support setting it up. It expects a single port (or none) for each server. It also doesn't support setting it up using the option passtrough directive.

[xkpx]

Thanks for the detailed explanation ! + karma

[meyergru]

I wonder if this has been asked before: is it possible to have HAProxy not expose a certificate in case no correct Host: header has been supplied?

Currently, if somebody scans my IP with just "openssl s_client <myip>:443", the default certificate(s) will be exposed. Going from there, one could enumerate every (sub-)domain that is presented with that certificate in order to find vulnerabilities.

If the certificate was instead withheld, one had to know at least one valid name in order to do that.

I imagine that e.g. with Cloudflare tunnels, a small range of IPs are being used to terminate all tunnel connections, but only after the Host: header has been presented will a specific customer certificate be chosen and presented, so that there is no possibilty for a "scan". I would like to do something similar with HAProxy on my OpnSense.

[jonf]

Hi,

I followed this tutorial and my services now work as intended by typing https://[service].[hostname] for each web service I have (in Docker containers). Now I have a Docker called Organizr installed which I want to act as a 'homepage' that displays links to those services, and I want this to be accessible by just typing https://[hostname] and still use the same Let's Encrypt certificate to secure it.

How can I do this within the framework of this setup?

[TheHellSite]

I wonder if this has been asked before: is it possible to have HAProxy not expose a certificate in case no correct Host: header has been supplied?

Currently, if somebody scans my IP with just "openssl s_client <myip>:443", the default certificate(s) will be exposed. Going from there, one could enumerate every (sub-)domain that is presented with that certificate in order to find vulnerabilities.

If the certificate was instead withheld, one had to know at least one valid name in order to do that.

I imagine that e.g. with Cloudflare tunnels, a small range of IPs are being used to terminate all tunnel connections, but only after the Host: header has been presented will a specific customer certificate be chosen and presented, so that there is no possibilty for a "scan". I would like to do something similar with HAProxy on my OpnSense.

Yes sure this is possible. In fact I am doing this for a long time already. Just didn't bother adding it to the guide.

I will add it as part 8 to the guide. Just give me a little time!

EDIT: Added this as Part 8.

[jonf]

Hi,

I followed this tutorial and my services now work as intended by typing https://[service].[hostname] for each web service I have (in Docker containers). Now I have a Docker called Organizr installed which I want to act as a 'homepage' that displays links to those services, and I want this to be accessible by just typing https://[hostname] and still use the same Let's Encrypt certificate to secure it.

How can I do this within the framework of this setup?

OK I tried by adding my full hostname as a new entry to the sub-domains mapping file followed by the corresponding backend for the Organizr service. When I type the hostname without https:// it doesn't load any SSL certificate or try to auto-redirect to HTTPS.  If I manually type https:// with the hostname, it gives an 'ERR_SSL_VERSION_OR_CIPHER_MISMATCH' error in my browser.

[meyergru]

Yes sure this is possible. In fact I am doing this for a long time already. Just didn't bother adding it to the guide.

I will add it as part 8 to the guide. Just give me a little time!

EDIT: Added this as Part 8.

Thank you very much, I just added it - works like a charm!

[xkpx]

Everything works for me but i tried to check some logs and saw something, Is this normal output when enable Detailed logging in 1_HTTPS_frontend, also if this is expected is there a way to keep the logging and disable this output somehow ?

Code: [Select]

2023-03-07T20:39:05 Error haproxy 192.168.1.102:47192 [07/Mar/2023:20:39:05.886] 1_HTTPS_frontend~ ADGUARD_backend/ADGUARD_server 0/-1/-1/-1/0 -1 0 - - CR-- 2/1/0/0/0 0/0 "GET https://adguard.xxx.yyy/assets/favicon.png HTTP/2.0"
2023-03-07T20:35:04 Error haproxy 3.252.130.46:37184 [07/Mar/2023:20:35:04.759] 1_HTTPS_frontend~ 1_HTTPS_frontend/<NOSRV> -1/-1/-1/-1/0 503 217 - - SC-- 4/2/0/0/0 0/0 "GET / HTTP/1.0"
2023-03-07T20:31:32 Error haproxy 192.168.1.102:47192 [07/Mar/2023:20:31:32.228] 1_HTTPS_frontend~ PROXMOX_backend/PROXMOX_server 0/0/0/-1/2 -1 0 - - CD-- 2/1/0/0/0 0/0 "GET https://proxmox.xxx.yyy/pve2/ext6/theme-crisp/resources/images/grid/hd-pop.png HTTP/2.0"
2023-03-07T20:31:03 Error haproxy 192.168.1.102:47192 [07/Mar/2023:20:31:03.640] 1_HTTPS_frontend~ PROXMOX_backend/PROXMOX_server 0/0/0/3/3 501 197 - - ---- 2/1/0/0/0 0/0 "GET https://proxmox.xxx.yyy/images/logo-ceph.png HTTP/2.0"

2023-03-07T20:31:00 Error haproxy 192.168.1.102:47212 [07/Mar/2023:20:31:00.634] 1_HTTPS_frontend/127.4.4.3:443: SSL handshake failure
2023-03-07T20:31:00 Error haproxy 192.168.1.102:47217 [07/Mar/2023:20:31:00.659] 1_HTTPS_frontend/127.4.4.3:443: SSL handshake failure
2023-03-07T20:31:00 Error haproxy 192.168.1.102:47214 [07/Mar/2023:20:31:00.649] 1_HTTPS_frontend/127.4.4.3:443: SSL handshake failure
And can someone post some implementation screenshots of NOSSLservice_rule

[Grenen]

First of all, thanks for a awesome guide!

In 5.8 you refer to a "FAQ about Map Files". Could you please link me to this FAQ? Perhaps it could be added as a link in the post so ppl easier can find it.

Second comes my question.
Ive finished the setup. I sorted out a *.-certificate for mydomain.com and added a A-record in the DNS for the domain with a homeassistant.mydomain.com that points to my public IP.
In HAProxy ive added a real server HA_server which points to the IP of my HA-server with port 8123. SSL checked.
I've added a backend pool HA_backend that points to my HA_server
Ive added a host override for host homeassistant domain mydomain.com with the internal IP of my HA-server.

My Local MAP-filehttps://ibb.co/D8xwgH5
My Public MAP-filehttps://ibb.co/D8xwgH5

When browsing to homeassistant.mydomain.com i get a "Unable to connect" message.

What did i do wrong, any tips where i should start looking?

[TheHellSite]

Everything works for me but i tried to check some logs and saw something, Is this normal output when enable Detailed logging in 1_HTTPS_frontend, also if this is expected is there a way to keep the logging and disable this output somehow ?

Code: [Select]

2023-03-07T20:39:05 Error haproxy 192.168.1.102:47192 [07/Mar/2023:20:39:05.886] 1_HTTPS_frontend~ ADGUARD_backend/ADGUARD_server 0/-1/-1/-1/0 -1 0 - - CR-- 2/1/0/0/0 0/0 "GET https://adguard.xxx.yyy/assets/favicon.png HTTP/2.0"
2023-03-07T20:35:04 Error haproxy 3.252.130.46:37184 [07/Mar/2023:20:35:04.759] 1_HTTPS_frontend~ 1_HTTPS_frontend/<NOSRV> -1/-1/-1/-1/0 503 217 - - SC-- 4/2/0/0/0 0/0 "GET / HTTP/1.0"
2023-03-07T20:31:32 Error haproxy 192.168.1.102:47192 [07/Mar/2023:20:31:32.228] 1_HTTPS_frontend~ PROXMOX_backend/PROXMOX_server 0/0/0/-1/2 -1 0 - - CD-- 2/1/0/0/0 0/0 "GET https://proxmox.xxx.yyy/pve2/ext6/theme-crisp/resources/images/grid/hd-pop.png HTTP/2.0"
2023-03-07T20:31:03 Error haproxy 192.168.1.102:47192 [07/Mar/2023:20:31:03.640] 1_HTTPS_frontend~ PROXMOX_backend/PROXMOX_server 0/0/0/3/3 501 197 - - ---- 2/1/0/0/0 0/0 "GET https://proxmox.xxx.yyy/images/logo-ceph.png HTTP/2.0"

2023-03-07T20:31:00 Error haproxy 192.168.1.102:47212 [07/Mar/2023:20:31:00.634] 1_HTTPS_frontend/127.4.4.3:443: SSL handshake failure
2023-03-07T20:31:00 Error haproxy 192.168.1.102:47217 [07/Mar/2023:20:31:00.659] 1_HTTPS_frontend/127.4.4.3:443: SSL handshake failure
2023-03-07T20:31:00 Error haproxy 192.168.1.102:47214 [07/Mar/2023:20:31:00.649] 1_HTTPS_frontend/127.4.4.3:443: SSL handshake failure
And can someone post some implementation screenshots of NOSSLservice_rule

This happens when the client doesn't support all the ciphers or uses Part 8 (if configured) to connect.

[TheHellSite]

First of all, thanks for a awesome guide!

In 5.8 you refer to a "FAQ about Map Files". Could you please link me to this FAQ? Perhaps it could be added as a link in the post so ppl easier can find it.

Second comes my question.
Ive finished the setup. I sorted out a *.-certificate for mydomain.com and added a A-record in the DNS for the domain with a homeassistant.mydomain.com that points to my public IP.
In HAProxy ive added a real server HA_server which points to the IP of my HA-server with port 8123. SSL checked.
I've added a backend pool HA_backend that points to my HA_server
Ive added a host override for host homeassistant domain mydomain.com with the internal IP of my HA-server.

My Local MAP-filehttps://ibb.co/D8xwgH5
My Public MAP-filehttps://ibb.co/D8xwgH5

When browsing to homeassistant.mydomain.com i get a "Unable to connect" message.

What did i do wrong, any tips where i should start looking?

You didn't read the tutorial properly. Read it again from the very top to the very bottom. Everything you just asked is answered there. Also if you would have followed the tutorial correctly you wouldn't see any errors now.

[Grenen]

You didn't read the tutorial properly. Read it again from the very top to the very bottom. Everything you just asked is answered there. Also if you would have followed the tutorial correctly you wouldn't see any errors now.

Ah, missunderstood the text regarding the map FAQ. Now i understand what you refered to.

Regarding my setup. Before i asked for help ive already gone through it twice. Ive now been through it twice again and i cant find anything wrong. I havent used virtual IP and instead using 127.0.0.1 as IP in the cases where you use your virtual IP.

Perhaps its working and Homeassistant is the problem? I found some threads regarding adding trusted proxies there. Anyone had the same problem and know what to configure in HA?

[TheHellSite]

You didn't read the tutorial properly. Read it again from the very top to the very bottom. Everything you just asked is answered there. Also if you would have followed the tutorial correctly you wouldn't see any errors now.

Ah, missunderstood the text regarding the map FAQ. Now i understand what you refered to.

Regarding my setup. Before i asked for help ive already gone through it twice. Ive now been through it twice again and i cant find anything wrong. I havent used virtual IP and instead using 127.0.0.1 as IP in the cases where you use your virtual IP.

Perhaps its working and Homeassistant is the problem? I found some threads regarding adding trusted proxies there. Anyone had the same problem and know what to configure in HA?

If you really did the guide four times now then I am suprised that you still don't know how to ask for help. 
Hint: I describe it in the first post.

[Grenen]

If you really did the guide four times now then I am suprised that you still don't know how to ask for help. 
Hint: I describe it in the first post.

Relevant question. I focused on the guide part, not the text above sorry.

Here is the config export.

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: 0_SNI_frontend (Listening on 0.0.0.0:80, 0.0.0.0:443)
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp
    default_backend SSL_backend

    # logging options

# Frontend: 1_HTTP_frontend (Listening to localhost)
frontend 1_HTTP_frontend
    bind 127.0.0.1:80 name 127.0.0.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor

    # logging options
    # ACL: NoSSL_condition
    acl acl_64025b0cc7a716.63164065 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_64025b0cc7a716.63164065

# Frontend: 1_HTTPS_frontend (Listening on 127.0.0.1:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.0.0.1:443 name 127.0.0.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/64025f85730fc6.50514236.certlist
    mode http
    option http-keep-alive
    option forwardfor
    timeout client 15m

    # logging options

    # ACTION: PUBLIC_SUBDOMAINS_rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/64025bea0b8443.12301363.txt)]

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server SSL_server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: BITWARDEN_backend ()
backend BITWARDEN_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server BITWARDEN_server 192.168.100.161:443 ssl verify none

# Backend: HA_backend ()
backend HA_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server HA_server 192.168.1.106:8123



# statistics are DISABLED

In the log file if i try to connect from the outside i get:

2023-03-08T09:49:15   Informational   haproxy   Connect from -external ip-:8335 to -public ip:443 (1_HTTPS_frontend/HTTP)   
2023-03-08T09:49:13   Informational   haproxy   Connect from -external ip-:8335 to -public ip-:443 (1_HTTPS_frontend/HTTP)   
2023-03-08T09:49:11   Informational   haproxy   Connect from -external ip-8335 to -public ip-:443 (1_HTTPS_frontend/HTTP)   
2023-03-08T09:49:10   Informational   haproxy   Connect from -external ip-:8335 to -public ip-:443 (1_HTTPS_frontend/HTTP)   
2023-03-08T09:49:10   Informational   haproxy   Connect from -external ip-:8335 to -public ip-:443 (0_SNI_frontend/TCP)   
2023-03-08T09:49:10   Informational   haproxy   Connect from -external ip-:8438 to -public ip-:443 (1_HTTPS_frontend/HTTP)   
2023-03-08T09:49:09   Informational   haproxy   Connect from -external ip-:8438 to -public ip8:443 (0_SNI_frontend/TCP)   
2023-03-08T09:49:09   Informational   haproxy   Connect from -external ip-:8430 to -public ip:443 (1_HTTPS_frontend/HTTP)   
2023-03-08T09:49:09   Informational   haproxy   Connect from -external ip-:8430 to -public ip:443 (0_SNI_frontend/TCP)

[TheHellSite]

Is the access to your bitwarden working?
If yes, is it working from WAN and LAN via haproxy? Or just WAN?

Please post a screenshot of your Unbound DNS overrides. You can blank out the domain name.