Author Topic: HowTo - Let's encrypt with HaProxy with 19.1.4 (Read 25243 times)

astromeier · « **on:** March 20, 2019, 07:43:58 pm »

TheHellSite made an actual V21.1 HowTo

After having troubles with my first installation I removed it completely, wiped all residual data and restarted from scratch.
Here I will describe what I did:

First a short description of my hardware installation:
A cable modem is directly connected to WAN interface - the WAN-Address comes via DHCP and changes with each restart of the modem
On the LAN side there is my home network with several PC clients and a RX300S6 server hosting several VMs.
All managed via DHCP and a fixed MAC-IP address configuration.
These VMs are different servers with Homepages (NGINX/Grav) and a cloud instance (NGINX/Nextcloud).
These VM servers should be accessible from WAN via HTTPS.
A HTTP request of a FQDN should be redirected (code 301) to HTTPS address.
The HaProxy offloads the HTTPS and dispatches the requests from WAN to the corresponding server via HTTP.
My FQDNs are CNAMED to a DynDNS service, whitch points to my actual WAN address.
The Dynamic DNS plugin updates the DynDNS service when the WAN address changes.
The Let's encrypt plugin keeps an eye to the certificates for HaProxy / Offloading.

My OPNsense configuration:

OPNsense 19.1.4-amd64 - FreeBSD 11.2-RELEASE-p9-HBSD - OpenSSL 1.0.2r 26 Feb 2019 - plain IPv4
HTTPS port for LAN accessed Administration GUI changed from 443 to another port, listen interface LAN
Enabled secure shell, port changed to another value, listen interface LAN & serial port enabled
other configs @ default values
installed plugins and chronology: 1. DynDNS, 2. HaProxy 3. Let's encrypt
single gateway @ WAN
Trust: No additional authorities or certificates for LE needed - they are installed by the LE plugin
WAN config: Block private and bogon network both activated
FW: No special rules for LAN
FW: WAN: HTTP (P:80) and HTTPS (P:443) pass any source to any destination - default gateway
No special NAT for HaProxy or LE plugin
FOR the next steps i urgendly recommend for all names etc: Don't use blanks, use an underline (e.g not "my name" but "my_name"
Plugings/HaProxy: One real server entry @ HTTP (P:80) for each VM server with a WAN accessible service and LAN internal HTTP communication
P/HAP: Backend Pools: One Backend for each real server - no rules
Then configure the Let's encrypt plugin: Settings: use stage environment for your first trials, check auto renewal and HaProxy integration
create an account
create a validation method: HaProxy HTTP integration and HTTP-01 challenge type, HaProxy HTTP frontend
create an automation: Restart HaProxy
create certificate entries for each FDQN: Common name is your FDQN like www.mydomain.de, Alt name e.g. mydomain.de and automation: restart HaProxy (all FDQN and Alt names must be active @ WAN!)
Back to LE/settings: Test config and activate plugin to make the HaProxy additions by LE-plugin
Back to Haproxy plugin:
P/HAP: Conditions: For each FDQN a "host matches" condition
One "no_HTTPS" condition: "SSL/TLS connection established" and this negated (Hook @ bottom of menu)
One "no_ACME_challenge" condition: Copy the "find_acme_challenge" (made by LE-plugin) to a new and negate this (Hook @ bottom of menu)
P/HAP: Advanced/Error messages: Create an Error Message for request that do not fit any FDQN - Code 403, content:
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
P/HAP: Rules: Make one rule per server/backend: If condition FDQN matches use specified Backend pool
Make one rule for HTTP to HTTPS redirection: If no_acme_challenge AND no_HTTPS do http-request-redirect with HTTP redirect "scheme https code 301"
Check if "redirect_acme_challenges" is present (made by LE plugin
Make a "no_host_match" rule: Test type Unless all of your FDQN match rules incl. the find_acme_challenge rule with an OR condition and execute function "http-request-deny"
Virtual services / Public Services: One Frontend for HTTP (P:80) :
Listen address: 0.0.0.0:80, HTTP Frontend default backend: acme_challenge_be
Rules: no_host_match, redirect_acme_challenge, http_redirect (to HTTPS)
Error message: deny_403 if nothing matches
and one for HTTPS (P:443) but disabled since the first certificates are validated - then you can enable it.
Listen address: 0.0.0.0:443 - SSL offloading activated, HTTPS default backend one other BE of your choice
Rules: no_host_match, redirect_acme_challenge, all the redirections to your several servers,
Error message: deny_403 if nothing matches
Go to the shell (e.g. via SSH) and check wether the path:
/var/etc/acme-client/challenges/.well-known/acme-challenge/
exists. if not create it with root:wheel owner
Go to HaProxy plugin, test config and enable it.
Go to LE plugin / Certifikates and (check for working on the staging platform) and Issue the certificates
If successful, switch to productive platform and renew the certifikates
If successful go to Haproxy / HTTPS public service / frontend and add the new certificates, enable this public service
Restart HaProxy

Hopefully I've nothing forgotten!!!
If so - your reverse Proxy should work now and is secured by LE certificates....

Please add your infos and suggestions in this thread!
If my may is way too complicated pleas show your easy way - I'm keen on it!

stay secure!
Thomas

ctflora · « **Reply #1 on:** December 13, 2019, 10:33:10 pm »

Just wanted to reply thanking you for this, it's working great! I'm home-bound for a few weeks following spinal surgery, so I'm finally getting to spend some time on my home lab. Have been trying to get this working off and on for about a month now, but after completely blowing away my previous HAProxy config and following along here step by step, it was practically easy!

I only had a couple issues following along that I wanted to share with anyone else using this guide:
First was that I got a Test Config failure after creating the first front-end. Turns out that I had to add an HTML header to the <html> tag in the Advanced/Error Message/403_deny. Don't know if this is a new requirement or what, but it's a simple fix. Altogether, it looked like this:

Code: [Select]

HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/plain
Retry-After: 60

<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

Second, I wasn't exactly sure where to add the certs to the frontend (on the very last step). Turns out I got it on the first try by adding my new certs to SSL Offloading/Certificates. I did not need to make any changes to the Client Certificate Auth section (default=disabled) which is what confused me.

Again, thank you for the write-up! Nextcloud seems to be working great behind the reverse proxy, so it's on to Collabora and a whole lot of others!

cyrus104 · « **Reply #2 on:** January 13, 2021, 06:02:29 am »

This has been extremely useful to get Next Cloud setup using LE and HAproxy.

I did want to make one change to the HAproxy error message to have it render properly.

Code: [Select]

HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html
Retry-After: 60

<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

akhiltoor · « **Reply #3 on:** February 05, 2021, 05:06:28 pm »

Hello,

I am French so sorry if my English is not completely correct.

Following a post I saw on the French community, I think it might be good to image this tutorial with screenshots.

Why ? Because it is easier to understand on screenshots than on plain text for those who do not speak English well.

I'm from PfSense, and I find OPNsense so much more enjoyable to use. Unfortunately it is not possible to find good tutorials, like for example HAProxy / Lets Encrypt.

You are under no obligation to answer my request, but I think it could be interesting that communities can work together.

Thanks for reading this message

astromeier · « **Reply #4 on:** February 05, 2021, 08:17:12 pm »

Dear akhiltoor, thanks for this suggestion - it makes sense to revise my HowTo and add some hardcopies.
In the next month I'll switch to new hardware and V 21.1.
Then I'll rewrite the HowTo ...

akhiltoor · « **Reply #5 on:** February 06, 2021, 11:47:23 am »

Hello astromeier,

Thank you very much for answering me.

It's really great, I can't wait to see the result.

I wish you good luck for the writing and the formatting of this next HowTo

Have a nice week.

Re4mstr · « **Reply #6 on:** March 20, 2021, 06:50:13 pm »

Thank you for a great guide.

I have been set up, but I do have a question:

- Is it possible to create a wildcard certificate, and if so, how do I set this up in ACME?

astromeier · « **Reply #7 on:** March 22, 2021, 08:04:03 pm »

Hi Re4mstr,
I've used the "Alt names" entry in the Certificates Options:
See attached screenshot:
Base Certificate is www.m****e.de and "Alt Names" is m****e.de for subdomains.

hth

Re4mstr · « **Reply #8 on:** March 22, 2021, 09:45:41 pm »

Quote from: astromeier on March 22, 2021, 08:04:03 pm

Hi Re4mstr,
I've used the "Alt names" entry in the Certificates Options:
See attached screenshot:
Base Certificate is www.m****e.de and "Alt Names" is m****e.de for subdomains.

hth

Wouldn't having "domain.com" in base be exactly the same thing?

Currently I'm having subdomains in the alt-names. It kinda works, but more work than I feel is necessary.

astromeier · « **Reply #9 on:** March 23, 2021, 02:42:50 pm »

Quote

Wouldn't having "domain.com" in base be exactly the same thing?

Hm, not sure - I haven't tested that....

mhartgring · « **Reply #10 on:** March 29, 2021, 01:03:56 pm »

Currently working on getting this set up using opnsense 21.1.3_3, os-haproxy 3.0 and os-acme-client 2.4
Aka, I'm running 'latest'

One "no_HTTPS" condition: "SSL/TLS connection established" and this negated (Hook @ bottom of menu)
This condition type doesn't exist (anymore). I chose "Traffic is HTTP", and then NOT negated.

Issue found, not fully resolved: The order of the rules when creating the "Virtual services / Public Services" is very important.

See https://github.com/opnsense/plugins/issues/1000 & https://github.com/opnsense/plugins/issues/1925

~~Remaining problem:~~

Quote

[WARNING] 087/142215 (72748) : parsing [/usr/local/etc/haproxy.conf.staging:66] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 087/142215 (72748) : parsing [/usr/local/etc/haproxy.conf.staging:68] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.

Code: [Select]

63     # ACTION: redirect_acme_challenges                                          
 64     use_backend acme_challenge_backend if acl_605e1a292ddce1.15865947           
 65     # ACTION: rule_redirect_http_https                                          
 66     http-request redirect scheme https code 301 if !acl_6061a61b811977.92979045 acl_605e848931a523.42844100
 67     # ACTION: rule_no_host_match                                                
 68     http-request deny unless acl_605e858f8de9e6.77623103 || acl_605e85820c35d5.14821142 || acl_605e8558014977.72403767 || acl_605e1a292ddce1.15    865947
 69     # ERROR FILE: Unknown_FQDN                                                  
 70     errorfile 403 /tmp/haproxy/errorfiles/605e84df732dd0.89785772.txt

The redirect to https (line 66) is always run before the redirect to use_backend (line 64) as shown in the logs of the acme-client

Quote

2021-03-29T13:12:25 acme.sh[16275] ] nope.nope:Verify error:Fetching https://nope.nope/.well-known/acme-challenge/{challenge key}

The full haproxy.conf.staging can be found here: https://pastebin.com/bkbU0s88[/s]

HC · « **Reply #11 on:** April 18, 2021, 08:10:08 pm »

Hey there,

could you write the instructions without abbreviations? I'm going crazy right now to bring bitwarden online with a subdomain

I search and search my german interface to find out what you mean with P/HAP The rest is okay right now, although it would be more understandable with screenshots.

Thank you in advance
HC

nfugal · « **Reply #12 on:** April 27, 2021, 11:45:41 pm »

Quote from: astromeier on February 05, 2021, 08:17:12 pm

Dear akhiltoor, thanks for this suggestion - it makes sense to revise my HowTo and add some hardcopies.
In the next month I'll switch to new hardware and V 21.1.
Then I'll rewrite the HowTo ...

I'd love to see an updated version. I'd be happy to help test things out too.

Thanks for the writeup!

TheHellSite · « **Reply #13 on:** May 31, 2021, 01:12:16 pm »

I might be a new user but I have plenty of experience with this kind of stuff.
So I wrote something up in the last couple of days.

https://forum.opnsense.org/index.php?topic=23339

astromeier · « **Reply #14 on:** May 31, 2021, 11:18:46 pm »

WOW!
Great guide - thank you very much for sharing!
So I think I don't have to update my outdated guide - yours is more up to date and complete!

Author Topic: HowTo - Let's encrypt with HaProxy with 19.1.4 (Read 25243 times)

astromeier

HowTo - Let's encrypt with HaProxy with 19.1.4

ctflora

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

cyrus104

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

akhiltoor

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

astromeier

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

akhiltoor

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

Re4mstr

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

astromeier

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

Re4mstr

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

astromeier

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

mhartgring

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

HC

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

nfugal

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

TheHellSite

Re: HowTo - Let's encrypt with HaProxy with 19.1.4

astromeier

Re: HowTo - Let's encrypt with HaProxy with 19.1.4