Tutorial 2024/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

[phamd4]

Hello,

Attached is my screenshot on my desec

main domain name is example.com
sub domain is truenas.example.com
dns zone content (attached in screenshot)
dydns update followed your guide ( using custom dydns2) username is my domain password is the token

I do hope I provide the information you have asked for.

Thank you so much for your help

[TheHellSite]

Correct me if I'm wrong below...

This tells me that you own a real domain and that you are not just using a subdomain of "dedyn.io".
It also tells me that you are probably using a managed DNS account at desec. --> You imported the DNS Zone of your domain to desec so that you manage it with them.

Because of that you need your DynDNS client to interact with desec and have "yourdomainname.com" set as the target which should get updated.
In the tutorial I used "tutorial.dedyn.io" as the target which will then automatically create the necessary A record in the DNS Zone.
Since you have your own domain and also want to use it within haproxy and not just subdomains of it, you will have to set the target of the DynDNS update to "yourdomainname.com".

Then a CNAME record with Subname "*" and Content "yourdomainname.com". So all subdomains point to the IP that gets associated to "yourdomainname.com" by the DynDNS plugin. You already have this configured correctly.

The thing you need to fix is the DynDNS part! There is something misconfigured.
Also check the logs! Maybe you copied the wrong credentials / access tokens from desec into the DynDNS config.

Part 2 - Step 8: username and hostname need to be "yourdomainname.com".

[phamd4]

Hello sir,

Attached is my DDNS configuration.

I think I understand what you are saying a little bit. So what I configured is right but you afraid my DDNS doesn't push the right update to my desec domain correct? I think i did it right since I have username is my domain name i.e. "example.com" password is my token. then hostname is just my domain name which is "example.com" correct? or do you want me to have it at *.example.com?

The weird thing is everything were working well. It just suddenly doesn't resolve if i just type in example.com. I have to use mysubdomain.example.com to get it working.

[opnuser1]

Yes, this shows me your haproxy export, but it doesn't tell me wether this is working for you like intended or not.

Also why do you have a "NoSSL_condition" and why did you link it to the serviceX_rules of the HTTP_frontend?
Remove it, this is totally unecessary and I never said that you need this.

Thank you.  I removed the NoSSL condition.

I believe everything is working as far as I can tell.  I will list how I tested.

I tested from a cell connection all the addresses, and everything is going to the right places and the sites load fine and work fine.  So, mostly this confirms that the SNI frontend is working perfectly I think.

Is Http frontend working?  I believe so, hard for me to confirm.  Nothing is really using the http traffic (don't get mad, yes I requested it, hear me out).  So I used it very briefly to set up that website that uses the install script on port 80 initially.  And during the setup, I saw in the Counters area of HAproxy that traffic went through the http frontend.  So that confirms that port 80 is working I think, and the site got successfully set up. 

So i think everything is working, and it is quite a nice setup.  I can add multiple reverse proxies with this setup, all on a single ip, and all the proxies can get ports 80 and 443 going there cleanly.

here is my updated config:

Code: [Select]

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         0.0.0.0 local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: SNI_frontend ()
frontend SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp

    # logging options
    option tcplog
    # ACL: TCP_SSL_condition
    acl acl_644c56b6785678.47181279 req.ssl_hello_type 1
    # ACL: TCP_server1_condition
    acl acl_644c5700ee7657.09485748 req.ssl_sni -m sub -i domain1.com
    # ACL: TCP_server2_condition
    acl acl_644c5719768e71.87060950 req.ssl_sni -i domain2.com

    # ACTION: TCP_RequestInspectDelay_rule
    # NOTE: actions with no ACLs/conditions will always match
    tcp-request inspect-delay 5s
    # ACTION: TCP_RequestContentAccept_rule
    tcp-request content accept if acl_644c56b6785678.47181279
    # ACTION: TCP_SERVICE1_rule
    use_backend TCP_SERVICE1_backend if acl_644c5700ee7657.09485748
    # ACTION: TCP_SERVICE2_rule
    use_backend TCP_SERVICE2_backend if acl_644c5719768e71.87060950

# Frontend: HTTP_frontend ()
frontend HTTP_frontend
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp

    # logging options
    option tcplog
    # ACL: http_server1_condition
    acl acl_6457247ca14984.71641345 hdr_sub(host) -i domain1.com
    # ACL: http_server2_condition
    acl acl_64572496aeac32.73416688 hdr_sub(host) -i domain2.com

    # ACTION: http_server1_rule
    use_backend TCP_SERVICE1_backend if acl_6457247ca14984.71641345
    # ACTION: http_server2_rule
    use_backend TCP_SERVICE2_backend if acl_64572496aeac32.73416688

# Backend: TCP_SERVICE1_backend ()
backend TCP_SERVICE1_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server TCP_SERVICE1_server 192.168.1.234

# Backend: TCP_SERVICE2_backend ()
backend TCP_SERVICE2_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server TCP_SERVICE2_server 192.168.1.231



# statistics are DISABLED




[phamd4]

Hello,

Just letting you know everything worked now. I went back and delete and re-add the configuration in the ddns and it were able to update the ip address now.

I had the exact same configuration and uses the same token and it worked now but not before. I don't know what changed but it worked. THank you so much

Hello sir,

Attached is my DDNS configuration.

I think I understand what you are saying a little bit. So what I configured is right but you afraid my DDNS doesn't push the right update to my desec domain correct? I think i did it right since I have username is my domain name i.e. "example.com" password is my token. then hostname is just my domain name which is "example.com" correct? or do you want me to have it at *.example.com?

The weird thing is everything were working well. It just suddenly doesn't resolve if i just type in example.com. I have to use mysubdomain.example.com to get it working.

[TheHellSite]

Is Http frontend working?  I believe so, hard for me to confirm.  Nothing is really using the http traffic (don't get mad, yes I requested it, hear me out).  So I used it very briefly to set up that website that uses the install script on port 80 initially.  And during the setup, I saw in the Counters area of HAproxy that traffic went through the http frontend.  So that confirms that port 80 is working I think, and the site got successfully set up. 

I still stand by my opinion that port 80 is not necessary at all. But since you never posted/linked that super duper script that would require port 80 this will forever be a myth to me.

[opnuser1]

I still stand by my opinion that port 80 is not necessary at all. But since you never posted/linked that super duper script that would require port 80 this will forever be a myth to me.

I did link it!
here it is:
https://v4-docs.chevereto.com/guides/docker/#create-https-proxy
That's the page with the installation instructions.  The port 80 is used for the place where i linked or the next command after.  I think it is used where I linked which sets up nginx.

the actual code is on the github:
https://github.com/chevereto/docker

but here's another use case that I like:
lets say i am using nginx for subdomains for domain1.com

but for another domain2.com, I'd like to use traefik as the reverse proxy.  So then, again, I'd need port 80 443 for both servers.  Some reverse proxy software work better with certain apps.

[TheHellSite]

I did link it!
here it is:
https://v4-docs.chevereto.com/guides/docker/#create-https-proxy
That's the page with the installation instructions.  The port 80 is used for the place where i linked or the next command after.  I think it is used where I linked which sets up nginx.

I stand corrected in terms of not providing the script.
Still I have not yet seen any part that would make it need port 80! If this is only needed in order to obtain https certificates for the nginx web server than this is just a lazy part on your end. Should be very easy to switch from HTTP-01 to DNS-01 mode in the ACME tool for the certificate creation.

but for another domain2.com, I'd like to use traefik as the reverse proxy.  So then, again, I'd need port 80 443 for both servers.  Some reverse proxy software work better with certain apps.

This is a pretty strong statement without any proof!
It is not about wether one reverse proxy works better than the other. This is all a matter of are you able to configure a (or any) reverse proxy correctly so that the services behind it can be accessed without any issues.

Again your "need" for port in this scenario 80 is pretty outdated and IMO comes from a lack of knowledge about the requirements of the services you would like to run. Everyone nowadays aims to secure outside going connections using https. Some do this by redirecting any port 80 connection to port 443 (like my tutorial). Others keep port 80 right out closed.
Why? Because using a reverse proxy makes port 80 on WAN side totally overdue.
If the service is internally running on port "god knows" with or without https (who cares) just put it behind a reverse proxy and secure the outside going connection using its ability to SSL offload and present it one port 443 and a valid HTTPS certificate.

The only possible scenario where a reverse proxy would need port 80 for a (good) reason is because they want to obtain their certificates using the HTTP-01 challenge. This on the other hand is mostly because the DNS provider / domain registrar doesn't have a DNS API to support the DNS-01 challenge. This alone should get one thinking about the choice of a domain registrar in 2023.
So we are back to the, IMO, laziness of the person setting things up.

Whenever I deploy a new service I read the documentation and try to lock it down as much as possible. Opening only the minimum required ports of the features I actually need. I also always configure them with self-signed SSL certificates to secure the connection to and from the reverse proxy.



I hope you don't take any offense from my writing, I don't mean to judge how you do your things.
I just think your current setup is... a bit clumsy.

Nonetheless I am glad that everything is working now how you wanted it to.

[mauro]

503 Service Unavailable

Dear all,
at the moment I have my webserver onine with portforwarding and before move on with HAProxy I'm thinking to test it setting it up locally. To keep also setting simple and possibly easier I'm considering reverse proxy of port 80 only for test.example.com
In other terms (IP as reference):
LAN (192.168.1.0/24, LAN Address 192.168.1.1)--> |HAProxy| --> DMZ (webserver 192.168.2.0/24, server, 192.168.2.2)

The only achievement I reached so far when I try to browse test.example.com is

503 Service Unavailable
No server is available to handle this request

this is haproxy setup:

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    2
    hard-stop-after             60s
    no strict-limits
    maxconn                     100
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# Frontend: test_http (Test http)
frontend test_http
    bind 192.168.1.1:80 name 192.168.1.1:80
    mode http
    option http-keep-alive
    default_backend example_backend

    # logging options
    # ACL: kanboard_c
    acl acl_6452ce5a700492.11355253 hdr(host) -i test.example.com

    # ACTION: kanboard_r
    use_backend test_backend if acl_6452ce5a700492.11355253

# Backend: test_backend (example pool)
backend example_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server s1_server 192.168.2.1:80 proto h2

# statistics are DISABLED

Any hint?

Next step will be using SSL. the webapplication have individual SSL certificae which I think I can import in opnsense to set up HTTPS redirection. This will be next gig

I've flattened HAProxy few times and reset but I always end up with error 503

I checked the firewall LAN -> DMZ and I don't see anything blocking the connection..

Thanks and please let me know if I can provide more information

cheers

[TheHellSite]

503 Service Unavailable

Dear all,
at the moment I have my webserver onine with portforwarding and before move on with HAProxy I'm thinking to test it setting it up locally. To keep also setting simple and possibly easier I'm considering reverse proxy of port 80 only for test.example.com
In other terms (IP as reference):
LAN (192.168.1.0/24, LAN Address 192.168.1.1)--> |HAProxy| --> DMZ (webserver 192.168.2.0/24, server, 192.168.2.2)

The only achievement I reached so far when I try to browse test.example.com is

503 Service Unavailable
No server is available to handle this request

this is haproxy setup:

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    2
    hard-stop-after             60s
    no strict-limits
    maxconn                     100
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# Frontend: test_http (Test http)
frontend test_http
    bind 192.168.1.1:80 name 192.168.1.1:80
    mode http
    option http-keep-alive
    default_backend example_backend

    # logging options
    # ACL: kanboard_c
    acl acl_6452ce5a700492.11355253 hdr(host) -i test.example.com

    # ACTION: kanboard_r
    use_backend test_backend if acl_6452ce5a700492.11355253

# Backend: test_backend (example pool)
backend example_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server s1_server 192.168.2.1:80 proto h2

# statistics are DISABLED

Any hint?

Next step will be using SSL. the webapplication have individual SSL certificae which I think I can import in opnsense to set up HTTPS redirection. This will be next gig

I've flattened HAProxy few times and reset but I always end up with error 503

I checked the firewall LAN -> DMZ and I don't see anything blocking the connection..

Thanks and please let me know if I can provide more information

cheers

Honestly please just follow my tutorial. I will not provide support for something else here.
If you want to do it your way then just ask in the appropriate forum.
But I will say if you keep on testing your way you will need much more time.

If it is not working with my way you can simply disable the WAN firewall rule and re-enable the NAT portforward.
This way you can also test this.

[mauro]

hi @TheHellSite,

before posting here I posted in the general forum and considering nobody answered I thhought it was because there is a more specific tutorial.

Anyway, thanks for your help and your tutorial which I found very interesting and I'll give it a go asap with all features

Cheers

[opnuser1]

I hope you don't take any offense from my writing, I don't mean to judge how you do your things.
I just think your current setup is... a bit clumsy.

Nonetheless I am glad that everything is working now how you wanted it to.

No problem at all, I take it as you are trying to give helpful advice.

I'm no networking expert.  If I were a network genius like you I wouldn't be here, would I?  So not sure what you want me to do.  This is the way I've able to wrap my mind around and it works, and I quite like it lol.  I'm sure there are better ways!  Thank you very much.  I've been looking forward to this configuration for a while.

[brandorf]

I had this working before, but I accidentialy hosed the install and didn't have my config back up.  (Lesson learned).

Now the issue I'm having is getting a 503 no matter what I've tried so far.  Not sure where the issue is.

For example, trying to hit my domain vtt.*.com I get the correct cert to the browser, but still a 503, and here's all I see in the haproxy log.

Code: [Select]

2023-05-08T23:00:55-05:00 Informational haproxy 192.168.0.198:60994 [08/May/2023:23:00:52.860] 1_HTTPS_frontend~ VTT_backend/FoundryVTT_server 0/3032/-1/-1/3035 503 217 - - SC-- 2/1/0/0/3 0/0 "GET https://vtt.*.com/favicon.ico HTTP/2.0"
2023-05-08T23:00:52-05:00 Informational haproxy 192.168.0.198:60994 [08/May/2023:23:00:49.702] 1_HTTPS_frontend~ VTT_backend/FoundryVTT_server 0/3045/-1/-1/3048 503 217 - - SC-- 2/1/0/0/3 0/0 "GET https://vtt.*.com/ HTTP/2.0"

Here's the haproxy config.  I hope you can help me see what I'm not seeing.

Code: [Select]

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: 0_SNI_frontend (Listening on port 80, 443)
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp
    default_backend SSL_backend

    # logging options
    option tcplog

# Frontend: 1_HTTP_frontend (Listening on :80)
frontend 1_HTTP_frontend
    bind 127.0.0.1:80 name 127.0.0.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor

    # logging options
    option httplog
    # ACL: NoSSL_condition
    acl acl_645996ff1a8d85.67011734 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_645996ff1a8d85.67011734

# Frontend: 1_HTTPS_frontend (Listening on localhost:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.0.0.1:443 name 127.0.0.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/64599b1898e146.09447169.certlist
    mode http
    option http-keep-alive
    option forwardfor
    timeout client 15m

    # logging options
    option httplog

    # ACTION: PUBLIC_DOMAINS_rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/645997471e1f42.25745091.txt)]

# Backend: acme_challenge_backend (Added by ACME Client plugin)
backend acme_challenge_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server acme_challenge_host 127.0.0.1:43580

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server SSL_server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: PLEX_backend ()
backend PLEX_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server PLEX_server 192.168.0.197:32400 ssl verify none

# Backend: VTT_backend ()
backend VTT_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server FoundryVTT_server 192.168.0.197:30000 ssl verify none

# Backend: Homeassistant_backend ()
backend Homeassistant_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server Homeassistant_server 192.168.0.196:8123 ssl verify none


[TheHellSite]

I had this working before, but I accidentialy hosed the install and didn't have my config back up.  (Lesson learned).

Now the issue I'm having is getting a 503 no matter what I've tried so far.  Not sure where the issue is.

For example, trying to hit my domain vtt.*.com I get the correct cert to the browser, but still a 503, and here's all I see in the haproxy log.

Code: [Select]

2023-05-08T23:00:55-05:00 Informational haproxy 192.168.0.198:60994 [08/May/2023:23:00:52.860] 1_HTTPS_frontend~ VTT_backend/FoundryVTT_server 0/3032/-1/-1/3035 503 217 - - SC-- 2/1/0/0/3 0/0 "GET https://vtt.*.com/favicon.ico HTTP/2.0"
2023-05-08T23:00:52-05:00 Informational haproxy 192.168.0.198:60994 [08/May/2023:23:00:49.702] 1_HTTPS_frontend~ VTT_backend/FoundryVTT_server 0/3045/-1/-1/3048 503 217 - - SC-- 2/1/0/0/3 0/0 "GET https://vtt.*.com/ HTTP/2.0"

Here's the haproxy config.  I hope you can help me see what I'm not seeing.

Code: [Select]

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: 0_SNI_frontend (Listening on port 80, 443)
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp
    default_backend SSL_backend

    # logging options
    option tcplog

# Frontend: 1_HTTP_frontend (Listening on :80)
frontend 1_HTTP_frontend
    bind 127.0.0.1:80 name 127.0.0.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor

    # logging options
    option httplog
    # ACL: NoSSL_condition
    acl acl_645996ff1a8d85.67011734 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_645996ff1a8d85.67011734

# Frontend: 1_HTTPS_frontend (Listening on localhost:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.0.0.1:443 name 127.0.0.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/64599b1898e146.09447169.certlist
    mode http
    option http-keep-alive
    option forwardfor
    timeout client 15m

    # logging options
    option httplog

    # ACTION: PUBLIC_DOMAINS_rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/645997471e1f42.25745091.txt)]

# Backend: acme_challenge_backend (Added by ACME Client plugin)
backend acme_challenge_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server acme_challenge_host 127.0.0.1:43580

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server SSL_server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: PLEX_backend ()
backend PLEX_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server PLEX_server 192.168.0.197:32400 ssl verify none

# Backend: VTT_backend ()
backend VTT_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server FoundryVTT_server 192.168.0.197:30000 ssl verify none

# Backend: Homeassistant_backend ()
backend Homeassistant_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server Homeassistant_server 192.168.0.196:8123 ssl verify none


Your HAProxy config looks good.
And since you get the 503 error this means connections are getting to HAProxy.
Your issue is mostlikely the SSL (yes or no) checkbox in the Real Server settings. Verify which service needs SSL and which doesn't for the local access between HAProxy and the local service.

Also unless you get your certificates using the HTTP-01 method, which I doubt, since there is no rule for it on the HTTP_frontend remove do the following:
in the HAProxy plugin: delete the acme_challenge_backend and acme_challenge_host and all other haproxy entries auto generated by the ACME plugin.

in the ACME plugin: Go the the settings and disable the "HAProxy Integration", hit Apply.

[mauro]

Honestly please just follow my tutorial. I will not provide support for something else here.
If you want to do it your way then just ask in the appropriate forum.
But I will say if you keep on testing your way you will need much more time.

If it is not working with my way you can simply disable the WAN firewall rule and re-enable the NAT portforward.
This way you can also test this.

I there, I followed the suggestion and at the end of the process i have this 2 issues which I can't figure out:
1) Certificate is not valid. I also run the ssllab test and I received the same answer (rating T) showing certificate not trusted
2) again error 503 Service unavailable

Checking the HAProxy log, it shows:

Informational   haproxy   public_IP:9911 [09/May/2023:17:25:07.299] 2_HTTPS_frontend/127.4.4.3:443: SSL handshake failure

which I think I solved removing the SSL tick on the  real server set up. I have the apache virtual server only listening on port 80

#1: is it possible it is because at the moment I'm using a staging cert?

#2: this is the issue I'm investigating now for few days without any luck. I'll go over your tutorial but hints are welcome

cheers