Tutorial 2024/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

[TheHellSite]

Essentially my question was if I could bind the SNI front end internally to a custom port instead of the usual 80,443.

First off all: B in your picture. The HTTP frontend shouldn't have the SSL_backend as the backend!!! Instead you only place the HTTPtoHTTPS redirect rule on it and if you need it, the HTTP services that don't require SSL.
You are also missing the PLEX_backend containing the PLEX_server.


Short Answer: No, because then you would have to access your services using that custom port in addition to having a subdomain for them. Which is pretty pointless, imho.

Example with default http/https port = http(s)://plex.yourdomain.tld/
Example with custom Port = 1234 = http(s)://plex.yourdomain.tld:1234/


Long Answer: It might be possible if you create a port forward in your OPNsense that forwards any WAN traffic that hits on TCP 80/443 to 127.0.0.1:1234.



My question to you: Why do you want't to do this in the first place? It makes this setup even more complex, for no reason...

[cookiemonster]

Thank you for the clarifications. They help. I'm revising it now.

[alexdelprete]

Hello,

when I started implementing HAProxy in my network I couldn't find any complete and well written guide out there. I had to puzzle everything together from various websites.
So I thought I would save many of you a lot of time and provide my ultimate HAProxy on OPNsense guide.  :)

This tutorial will show you how to configure HAProxy as a reverse proxy on OPNsense using wildcard certificates from Let's Encrypt.
It is going to be a step-by-step guide with images on how to set things up while also explaining why we set things up in a certain way.
I will try to make this as complete and detailed as possible.
If you think that there is anything wrong or missing, feel free to tell me about it and I will consider changing it.

If this guide was helpful to you then please leave me a thanks down below as it took me several days to write this down.

Kind Regards
TheHellSite

This guide was a life-saver. And while implementing, I finally had the chance to understand a little more how HAProxy works. Thank you so much for writing it.

Everything worked as documented, and in a couple of hours I was up and running, the only problem was getting 100% on the ssl quality test. The Cipher Strength was always 90%, so I got A+, with a 90% overall score.

After a couple of hours of tinkering with the ciphers, I figured out that the test doesn't like 128bit ciphers. So I removed those from the Cipher List (TLSv1.2) and Cipher Suites (TLSv1.3) of the HTTPS_frontend, and I finally got the A+ 100% score.

In case anyone faces the same issue, these are the cipher settings for HTTPS_frontend:

Cipher List: ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305

Cipher Suites: TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256

Thanks again,

Alessandro

[TheHellSite]

After a couple of hours of tinkering with the ciphers, I figured out that the test doesn't like 128bit ciphers. So I removed those from the Cipher List (TLSv1.2) and Cipher Suites (TLSv1.3) of the HTTPS_frontend, and I finally got the A+ 100% score.

In case anyone faces the same issue, these are the cipher settings for HTTPS_frontend:

Cipher List: ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305

Cipher Suites: TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256

Thanks again,

Alessandro

Happy to hear it is working for you.
I think you missed the part of my tutorial where I am giving needed ciphers and cipher suites to get an 100% A+ rating.

My Tutorial
Cipher List: ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384
Cipher Suites: TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256

yours
Cipher List: ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305
Cipher Suites: TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256

The only difference between my guide and your list is the "DHE-RSA-AES256-GCM-SHA384" cipher.
Both will be scoring 100 % A+ while mine offer even more client compatibility.

Seems like I forgot to mention it again during tutorial at the part where the HTTPS_frontend is created. I now put a reference in there pointing to the beginning of my post where I provide the current best cipher list and cipher suits.

[alexdelprete]

Happy to hear it is working for you.
I think you missed the part of my tutorial where I am giving needed ciphers and cipher suites to get an 100% A+ rating.

Indeed I missed it, even though I followed the guide with great attention. Problem is that it was not in the tutorial, but at the beginning of the post. Sorry about that, I paid attention mainly to the tutorial. Would have saved a lot of time tinkering with the ciphers. :)

The only difference between my guide and your list is the "DHE-RSA-AES256-GCM-SHA384" cipher.
Both will be scoring 100 % A+ while mine offer even more client compatibility.

Yes I noticed that single difference. I started from the default ciphers of HAProxy, and that one wasn't in there. I simply removed the AES128 ones from the defaults. The link you put to that  Mozilla Config is fantastic, never even thought something like that existed. Thank you again.

Seems like I forgot to mention it again during tutorial at the part where the HTTPS_frontend is created. I now put a reference in there pointing to the beginning of my post where I provide the current best cipher list and cipher suits.

That's good, because I missed it completely, would've saved me quite some time. My bad. :)

One thing I wanted to ask you: I followed your naming conventions and I noticed you had the 1_ prefix both for the HTTP and HTTPS frontend. I renamed the HTTPS to 2_HTTPS_frontend, don't know if it was intentional or not, but I interpreted it as a progressive number so that one was a 2.

[TheHellSite]

One thing I wanted to ask you: I followed your naming conventions and I noticed you had the 1_ prefix both for the HTTP and HTTPS frontend. I renamed the HTTPS to 2_HTTPS_frontend, don't know if it was intentional or not, but I interpreted it as a progressive number so that one was a 2.

I did that on purpose to express the "level" of reverse proxying.

Level 1 - SNI traffic
Level 2 - HTTP + HTTPS traffic

However you can name it as you like. It doesn't matter in terms of functionality.

[alexdelprete]

I did that on purpose to express the "level" of reverse proxying.

Level 1 - SNI traffic
Level 2 - HTTP + HTTPS traffic

However you can name it as you like. It doesn't matter in terms of functionality.

Thanks for the explanation.

[alexdelprete]

Seems like I forgot to mention it again during tutorial at the part where the HTTPS_frontend is created. I now put a reference in there pointing to the beginning of my post where I provide the current best cipher list and cipher suits.

I moved the frontend cipher settings and bind options to the Global Parameters -> SSL Default Settings section, emptied the relevant fields in HTTPS_frontend, it works fine and I think it's simpler:

[TheHellSite]

Arguable! ;)
If you have many HTTPS frontends (on different ports) that might need different SSL settings then my way is better.

Otherwise it doesn't really matter where you put the settings. Just note that the SSL default settings get overwritten once you set anything in the associated boxes on the HTTPS frontends.

[alexdelprete]

Arguable! ;)
If you have many HTTPS frontends (on different ports) that might need different SSL settings then my way is better.

Otherwise it doesn't really matter where you put the settings. Just note that the SSL default settings get overwritten once you set anything in the associated boxes on the HTTPS frontends.

Your second point is the answer to your first observation: Global settings are a "global default". If you need a custom config for a frontend, you can customize its config and it will supersede the global one. ;)

That's why I like it, it's the classic enterprise approach to organizing things, more intuitive for me (global...and custom as an exception).

But like you said...it's all very subjective obviously. Individual preferences.

[dima1002]

Hello,

what is wrong here?

In Firefox I get the following error: PR_END_OF_FILE_ERROR

Code Select Expand


#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    4
    hard-stop-after             60s
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log audit debug
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend (DISABLED): LetsEncrypt_443 ()

# Frontend (DISABLED): LetsEncrypt_80 ()

# Frontend: 1_HTTP_frontend ()
frontend 1_HTTP_frontend
    bind 127.0.0.1:80 name 127.0.0.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: NoSSL_condition
    acl acl_61a24897421141.86617043 req.ssl_ver gt 0

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_61a24897421141.86617043

# Frontend: 1_HTTPS_frontend ()
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.0.0.1:443 name 127.0.0.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256 ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/61a24a78aa9cc4.11915455.certlist
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options

    # ACTION: PUBLIC_SUBDOMAINS_map-rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/61a249350142e3.01879320.txt)]

# Frontend: 0_SNI_frontend ()
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443 accept-proxy
    bind 0.0.0.0:80 name 0.0.0.0:80 accept-proxy
    mode tcp
    default_backend SSL_backend
    # tuning options
    timeout client 30s

    # logging options

# Backend: Mail ()
backend Mail
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server Main 192.168.111.2:443 ssl verify none

# Backend (DISABLED): acme_challenge_backend (Added by Let's Encrypt plugin)

# Backend: Nextcloud ()
backend Nextcloud
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server Nextcloud 192.168.111.3:443 ssl verify none

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server SSL_Server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: Bitwarden_Backend ()
backend Bitwarden_Backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server Vaultwarden80 192.168.111.77:80


[TheHellSite]

Hello,

what is wrong here?

In Firefox I get the following error: PR_END_OF_FILE_ERROR

Are you only getting this error when using Firefox?
If so, which version of Firefox are you running?

[SOUK]

Delete

[evathesalmon]

In Firefox I get the following error: PR_END_OF_FILE_ERROR

Hello,

I have same issue in firefox and chrome. In logs it is said "Received something which does not look like a PROXY protocol header"



I thought at first that it is a proxy problem and double checked your 20210613 update but my bind option pass-through set as accept-proxy





My current HAProxy version is 3.7 and opnsense is 21.7.6

[alexdelprete]

In Part 6, NAT Reflection: it applies to port forwarding rules, but in the guide you switched to a simple filter rule.

So there's only one option remaining: split DNS.

HAProxy has been rock solid, thanks again for your guide. I'm having a hard time only for Uptime Kuma, it uses websockets, and it's the only service that doesn't work behind HAProxy. The dev published a guide for the configuration behind several reverse proxies, unfortunately the only one missing is HAProxy: https://github.com/louislam/uptime-kuma/wiki/Reverse-Proxy#nginx

I'm sure there's a way to make it work but I can't find it...