Tutorial 2024/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

[N0_Klu3]

Yes OPNsense is my only router/firewall.

I have dual WAN created a gateway group to double internet speed.
I created the rule on only 1 WAN gateway tho, and my a records point to the same IP.

I have 3 LANs (not VLANS) 10.0.0.1 (Main LAN) - 192.168.107.1 (IoT) - 192.168.200.1 (Guest)
My NAS Server (10.0.0.10) and OPNsense (10.0.0.1) are on 10.0.0.1/24 LAN, so no going through anything different there.

I wonder if with the dual WAN it needs a specific rule?
As with dual WAN I have to change the default gateway on the Allow LAN to any rule to the specific Multi_Gateway...
Like my LAN can no longer talk to my IOT Lan without having a specific rule to say allow...

I've just added the LAN Net to Loopback but still no dice

[TheHellSite]

The error is not related to your dual WAN setup.  ;D
I misread something in your HAProxy config...

Code Select Expand

server jellyfin 10.0.0.10:8090 check inter 2s port 8090  ssl verify none
You see that "ssl" word in your server config?
SSL and HTTP don't like each other!  ;)

Since your services are NOT using HTTPS but only HTTP locally... you need to UNCHECK the SSL checkbox in the HAProxy real server config for both of your servers.

[N0_Klu3]

The error is not related to your dual WAN setup.  ;D
I misread something in your HAProxy config...

Code Select Expand

server jellyfin 10.0.0.10:8090 check inter 2s port 8090  ssl verify none
You see that "ssl" word in your server config?
SSL and HTTP don't like each other!  ;)

Since your services are NOT using HTTPS but only HTTP locally... you need to UNCHECK the SSL checkbox in the HAProxy real server config for both of your servers.

Oh my days!!!
That worked! I spent all night trying to figure it out.

Thank you so very much!

[blackhand27]

Great Guide! Took me a bit since im using Godaddy for my domain name but figured out the cert mess and got my plex server working. Been hoping to get this setup so I can run a Synapse server at home.

I do seem to be having issues with my bitwarden site, I have the proper port set in HAProxy however I have a feeling its something with its configuration still using the internal cert vs the offload. Would anyone have some advice to see what I can look for?

figured it out, I reinstalled the server using self signed certs, for whatever reason it was still using the builtin lets encrypt certs and failing to connect.

[lilsense]

@TheHellSite

I think the problem is with the SNI frontend. Here the SSL backend is specified as the default backend. He doesn't even look at the MAP file. he forwards everything to the SSL backend. When I set the openvpn backend as default Backend for a test in the SNI frontend, openvpn work but the other things not.

do you have an idea how I can solve this?

I was just about to write you exactly this!  ;D
Your reply confirmed my guess.

Looking through the manual pages of HAProxy it seems that the "Default Backend" setting can only be overwritten by a "Use Backend" rule! Which a "Use map file" rule isn't able to.
https://www.haproxy.com/de/blog/the-four-essential-sections-of-an-haproxy-configuration/
But I can't imagine that this is the intended behaviour.

Anyways... you simply need to create a VPN_condition "host starts with vpn" and a "use backend OPENVPN_backend if VPN_condition=true" rule.
Add this rule to the SNI_frontend and set the default backend back to the SSL_backend.

Am I confused about this?

It looks like the current set up on page one will not work due to "default backend"??

[TheHellSite]

I was just about to write you exactly this!  ;D
Your reply confirmed my guess.

Looking through the manual pages of HAProxy it seems that the "Default Backend" setting can only be overwritten by a "Use Backend" rule! Which a "Use map file" rule isn't able to.
https://www.haproxy.com/de/blog/the-four-essential-sections-of-an-haproxy-configuration/
But I can't imagine that this is the intended behaviour.

Anyways... you simply need to create a VPN_condition "host starts with vpn" and a "use backend OPENVPN_backend if VPN_condition=true" rule.
Add this rule to the SNI_frontend and set the default backend back to the SSL_backend.

Am I confused about this?

It looks like the current set up on page one will not work due to "default backend"??

I am not sure what you mean?
The setup still works as described. I also got OpenVPN over TCP working but haven't had the time yet to add it to my tutorial.

[thegriffen]

Hi everyone

First of all.. Awesome guide and even so, that you update it with new stuff as it comes along.

Iv been kinda hitting my head for a day or two now and gotta throw in the towel, and put my troubles in here.

im running a small test setup, where im gonna have a couple of web services running, but the thing is here, they are running traefik with there own LE ssl validation and so. but i cant get HAproxy to work propper.

My config looking like this:

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    4
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log /var/run/log local0 debug

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: 0_SNI_Frontend ()
frontend 0_SNI_Frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind 0.0.0.0:8888 name 0.0.0.0:8888
    mode tcp
    default_backend SSL_Backend
    # tuning options
    timeout client 30s

    # logging options
    option tcplog

    # ACTION: Public_Domain_Map_Rule_2
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/6178606dd3b431.48344137.txt)]
    # ACTION: Public_Domain_Map_Rule_3
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/6178704b1d59f6.07557436.txt)]

# Frontend: 1_Http_Frontend ()
frontend 1_Http_Frontend
    bind 0.0.0.0:80 name 0.0.0.0:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    option httplog
    # ACL: NoSSL_Condition
    acl acl_61781026415d99.64392811 req.ssl_ver gt 0

    # ACTION: HttpToHttps_Rule
    http-request redirect scheme https code 301 if !acl_61781026415d99.64392811

# Frontend: 1_Https_Frontend ()
frontend 1_Https_Frontend
    bind 0.0.0.0:443 name 0.0.0.0:443 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 15m

    # logging options
    option httplog

    # ACTION: Public_Domain_Map_Rule_1
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/617812015049f9.00974692.txt)]

# Backend: SSL_Backend (Loopback)
backend SSL_Backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server SSL_Server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: Central_OpenSense_FW_Backend (OpenSense Firewal)
backend Central_OpenSense_FW_Backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server Central_OpenSense_FW_SRV 10.10.5.2:55443 ssl verify none

# Backend: FRCD_Backend (domain1.com)
backend FRCD_Backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server DMZ_domain1_SRV 10.10.20.40:443 ssl alpn h2,http/1.1,http/1.0 verify none

# Backend: FRCD_2_Backend (domain2:8888)
backend FRCD_2_Backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server DMZ_domain2_SRV 10.10.20.41:8888

Best regrads

[TheHellSite]

Hi everyone

First of all.. Awesome guide and even so, that you update it with new stuff as it comes along.

Iv been kinda hitting my head for a day or two now and gotta throw in the towel, and put my troubles in here.

im running a small test setup, where im gonna have a couple of web services running, but the thing is here, they are running traefik with there own LE ssl validation and so. but i cant get HAproxy to work propper.

I don't really understand what you need haproxy for if you are already using traefik?
Can you please elaborate further what...

1. You are trying to achieve by using HAProxy?

2. What issue you are facing with HAProxy right now?

3. Explain your local network layout further!

[bringha]

Hi there,

first of all a big big thank you for this awesome and comprehensive tutorial. Very helpfull and a great contribution.

I have an additional question and I am not sure whether I suffer from a big misuderstanding.

I configured my Dyndns as suggested with dedyn.io and have now a domain.dedyn.io properly working. Your tutorial now assumes to create wildcard certificates for the *.domain.dedyn.io (in my case)

I have a main domain registered with a poster somewhere else which is domain.com. Historically I reach my dyndns based subdomains via CNAME DNS entries at my main domain provider's DNS systems, eg home.domain.com points then to home.domain.dedyn.io.

It is now possible to let the acme client generate wildcard certificates also for *.domain.com accordingly in addition/replacing the wildcard certs for *.domain.dedyn.io?

Looking forward to your reply.

Br br

[TheHellSite]

I configured my Dyndns as suggested with dedyn.io and have now a domain.dedyn.io properly working. Your tutorial now assumes to create wildcard certificates for the *.domain.dedyn.io (in my case)

I have a main domain registered with a poster somewhere else which is domain.com. Historically I reach my dyndns based subdomains via CNAME DNS entries at my main domain provider's DNS systems, eg home.domain.com points then to home.domain.dedyn.io.

It is now possible to let the acme client generate wildcard certificates also for *.domain.com accordingly in addition/replacing the wildcard certs for *.domain.dedyn.io?

Basically there are two options to solve this

1. You replace "deSEC" as your DynDNS provider and use the DynDNS of your main domain registrar. If he supports this...
--> What is your domain registrar? (Where your real "domain.com" is registered...)

2. You transfer the DNS zone from your main "domain.com" to deSEC. Again, if your current domain registrar supports this.
This way your domain is still hosted at your current domain registrar, but deSEC will be managing all the DNS related stuff. (A records, ...)

[bringha]

Thank you for your answer!

My domain Registrar is Ionos. Ionos has also a dyndns service and a dns management api and is contained in the opnsense dyndns list. I would like to leave the services of the main Domain there (mail, Webhosting) and have only the subdomains with my local dyndns based site.

Br br

[TheHellSite]

IONOS works exactly like deSEC, just adopt the deSEC stuff to the IONOS settings.
get a dns API key, set up dyn dns and get your certs.

[N0_Klu3]

Hey @TheHellSite, Do you know a way I can add Basic Auth to one of the sub domains?

I have a dashboard which just runs without login and I would really like it secured behind just basic auth if possible.
I tried creating a condition and a rule and applying it to the sub domain but it doesnt seem to work.

I also saw on the backend there is an option on the domain for basic auth checkbox, so I tried that but no dice either.

Any chance you can add a section to the guide about securing singular domains with basic auth?

[TheHellSite]

Hey @TheHellSite, Do you know a way I can add Basic Auth to one of the sub domains?

I have a dashboard which just runs without login and I would really like it secured behind just basic auth if possible.
I tried creating a condition and a rule and applying it to the sub domain but it doesnt seem to work.

I also saw on the backend there is an option on the domain for basic auth checkbox, so I tried that but no dice either.

Any chance you can add a section to the guide about securing singular domains with basic auth?

Definitely will be looking into this. You can expect this in the coming weeks, hopefully.

[N0_Klu3]

AMAZING! Thank you so much