HowTo - Let's encrypt with HaProxy with 19.1.4

Started by astromeier, March 20, 2019, 07:43:58 PM

Previous topic - Next topic
Print
Go Down Pages1 2
User actions
March 20, 2019, 07:43:58 PM Last Edit: June 01, 2021, 10:57:39 PM by astromeier
TheHellSite made an actual V21.1 HowTo

After having troubles with my first installation I removed it completely, wiped all residual data and restarted from scratch.
Here I will describe what I did:

First a short description of my hardware installation:
A cable modem is directly connected to WAN interface - the WAN-Address comes via DHCP and changes with each restart of the modem
On the LAN side there is my home network with several PC clients and a RX300S6 server hosting several VMs.
All managed via DHCP and a fixed MAC-IP address configuration.
These VMs are different servers with Homepages (NGINX/Grav) and a cloud instance (NGINX/Nextcloud).
These VM servers should be accessible from WAN via HTTPS.
A HTTP request of a FQDN should be redirected (code 301) to HTTPS address.
The HaProxy offloads the HTTPS and dispatches the requests from WAN to the corresponding server via HTTP.
My FQDNs are CNAMED to a DynDNS service, whitch points to my actual WAN address.
The Dynamic DNS plugin updates the DynDNS service when the WAN address changes.
The Let's encrypt plugin keeps an eye to the certificates for HaProxy / Offloading.

My OPNsense configuration:

  • OPNsense 19.1.4-amd64 - FreeBSD 11.2-RELEASE-p9-HBSD - OpenSSL 1.0.2r 26 Feb 2019 - plain IPv4
  • HTTPS port for LAN accessed Administration GUI changed from 443 to another port, listen interface LAN
  • Enabled secure shell, port changed to another value, listen interface LAN & serial port enabled
  • other configs @ default values
  • installed plugins and chronology: 1. DynDNS, 2. HaProxy 3. Let's encrypt
  • single gateway @ WAN
  • Trust: No additional authorities or certificates for LE needed - they are installed by the LE plugin
  • WAN config: Block private and bogon network both activated
  • FW: No special rules for LAN
  • FW: WAN: HTTP (P:80) and HTTPS (P:443) pass any source to any destination - default gateway 
  • No special NAT for HaProxy or LE plugin
  • FOR the next steps i urgendly recommend for all names etc: Don't use blanks, use an underline (e.g not "my name" but "my_name"
  • Plugings/HaProxy: One real server entry @ HTTP (P:80) for each VM server with a WAN accessible service and LAN internal HTTP communication
  • P/HAP: Backend Pools: One Backend for each real server - no rules
  • Then configure the Let's encrypt plugin:  Settings: use stage environment for your first trials, check auto renewal and HaProxy integration
  • create an account
  • create a validation method: HaProxy HTTP integration and HTTP-01 challenge type, HaProxy HTTP frontend
  • create an automation: Restart HaProxy
  • create certificate entries for each FDQN: Common name is your FDQN like www.mydomain.de, Alt name e.g. mydomain.de and automation: restart HaProxy (all FDQN and Alt names must be active @ WAN!)
  • Back to LE/settings: Test config and activate plugin to make the HaProxy additions by LE-plugin
  • Back to Haproxy plugin: 
  • P/HAP: Conditions: For each FDQN a "host matches" condition
    One "no_HTTPS" condition: "SSL/TLS connection established" and this negated (Hook @ bottom of menu)
    One "no_ACME_challenge" condition: Copy the "find_acme_challenge" (made by LE-plugin) to a new and negate this  (Hook @ bottom of menu)
  • P/HAP: Advanced/Error messages: Create an Error Message for request that do not fit any FDQN - Code 403, content:
    <html><body><h1>403 Forbidden</h1>
    Request forbidden by administrative rules.
    </body></html>
  • P/HAP: Rules: Make one rule per server/backend: If condition FDQN matches use specified Backend pool
    Make one rule for HTTP to HTTPS redirection: If no_acme_challenge AND no_HTTPS do http-request-redirect with HTTP redirect "scheme https code 301"
    Check if "redirect_acme_challenges" is present (made by LE plugin
  • Make a "no_host_match" rule: Test type Unless all of your FDQN match rules incl. the find_acme_challenge rule with an OR condition and execute function "http-request-deny"
  • Virtual services / Public Services: One Frontend for HTTP (P:80) :
    Listen address: 0.0.0.0:80, HTTP Frontend default backend: acme_challenge_be
    Rules: no_host_match, redirect_acme_challenge, http_redirect (to HTTPS)
    Error message: deny_403 if nothing matches
    and one for HTTPS (P:443) but disabled since the first certificates are validated - then you can enable it.
    Listen address:  0.0.0.0:443  - SSL offloading activated, HTTPS default backend one other BE of your choice
    Rules: no_host_match, redirect_acme_challenge, all the redirections to your several servers,
    Error message: deny_403 if nothing matches
  • Go to the shell (e.g. via SSH) and check wether the path:
    /var/etc/acme-client/challenges/.well-known/acme-challenge/
    exists. if not create it with root:wheel owner
  • Go to HaProxy plugin, test config and enable it.
  • Go to LE plugin / Certifikates and (check for working on the staging platform) and Issue the certificates
  • If successful, switch to productive platform and renew the certifikates
  • If successful go to Haproxy / HTTPS public service / frontend and add the new certificates, enable this public service
  • Restart HaProxy

Hopefully I've nothing forgotten!!!
If so - your reverse Proxy should work now and is secured by LE certificates....

Please add your infos and suggestions in this thread!
If my may is way too complicated pleas show your easy way - I'm keen on it!

stay secure!
Thomas
Stay secure!
Thomas

OPNsense 22.x / Qotom Q370G4 ram8G ssd256G
December 13, 2019, 10:33:10 PM #1
Just wanted to reply thanking you for this, it's working great!  I'm home-bound for a few weeks following spinal surgery, so I'm finally getting to spend some time on my home lab.  Have been trying to get this working off and on for about a month now, but after completely blowing away my previous HAProxy config and following along here step by step, it was practically easy!

I only had a couple issues following along that I wanted to share with anyone else using this guide:
First was that I got a Test Config failure after creating the first front-end.  Turns out that I had to add an HTML header to the <html> tag in the Advanced/Error Message/403_deny.  Don't know if this is a new requirement or what, but it's a simple fix.  Altogether, it looked like this:

Code Select
HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/plain
Retry-After: 60

<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

Second, I wasn't exactly sure where to add the certs to the frontend (on the very last step).  Turns out I got it on the first try by adding my new certs to SSL Offloading/Certificates.  I did not need to make any changes to the Client Certificate Auth section (default=disabled) which is what confused me.

Again, thank you for the write-up!  Nextcloud seems to be working great behind the reverse proxy, so it's on to Collabora and a whole lot of others!
January 13, 2021, 06:02:29 AM #2
This has been extremely useful to get Next Cloud setup using LE and HAproxy.

I did want to make one change to the HAproxy error message to have it render properly.

Code Select

HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html
Retry-After: 60

<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
February 05, 2021, 05:06:28 PM #3
Hello,

I am French so sorry if my English is not completely correct.

Following a post I saw on the French community, I think it might be good to image this tutorial with screenshots.

Why ? Because it is easier to understand on screenshots than on plain text for those who do not speak English well.

I'm from PfSense, and I find OPNsense so much more enjoyable to use. Unfortunately it is not possible to find good tutorials, like for example HAProxy / Lets Encrypt.

You are under no obligation to answer my request, but I think it could be interesting that communities can work together.

Thanks for reading this message  :D
February 05, 2021, 08:17:12 PM #4
Dear akhiltoor, thanks for this suggestion - it makes sense to revise my HowTo and add some hardcopies.
In the next month I'll switch to new hardware and V 21.1.
Then I'll rewrite the HowTo ...
Stay secure!
Thomas

OPNsense 22.x / Qotom Q370G4 ram8G ssd256G
February 06, 2021, 11:47:23 AM #5
Hello astromeier,

Thank you very much for answering me.

It's really great, I can't wait to see the result.

I wish you good luck for the writing and the formatting of this next HowTo

Have a nice week.
March 20, 2021, 06:50:13 PM #6 Last Edit: March 22, 2021, 11:23:54 PM by Re4mstr
Thank you for a great guide.

I have been set up, but I do have a question:

- Is it possible to create a wildcard certificate, and if so, how do I set this up in ACME?
HP EliteDesk 800 G2 Mini - i5 3.2Ghz, 8GB RAM, 1TB SSD
March 22, 2021, 08:04:03 PM #7
Hi Re4mstr,
I've used the "Alt names" entry in the Certificates Options:
See attached screenshot:
Base Certificate is www.m****e.de and "Alt Names" is m****e.de for subdomains.

hth

Stay secure!
Thomas

OPNsense 22.x / Qotom Q370G4 ram8G ssd256G
March 22, 2021, 09:45:41 PM #8 Last Edit: March 22, 2021, 10:14:57 PM by Re4mstr
Quote from: astromeier on March 22, 2021, 08:04:03 PM
Hi Re4mstr,
I've used the "Alt names" entry in the Certificates Options:
See attached screenshot:
Base Certificate is www.m****e.de and "Alt Names" is m****e.de for subdomains.

hth

Wouldn't having "domain.com" in base be exactly the same thing?

Currently I'm having subdomains in the alt-names. It kinda works, but more work than I feel is necessary.
HP EliteDesk 800 G2 Mini - i5 3.2Ghz, 8GB RAM, 1TB SSD
March 23, 2021, 02:42:50 PM #9
QuoteWouldn't having "domain.com" in base be exactly the same thing?
Hm, not sure - I haven't tested that....
Stay secure!
Thomas

OPNsense 22.x / Qotom Q370G4 ram8G ssd256G
March 29, 2021, 01:03:56 PM #10 Last Edit: March 29, 2021, 02:55:01 PM by mhartgring
Currently working on getting this set up using opnsense 21.1.3_3, os-haproxy 3.0 and os-acme-client 2.4
Aka, I'm running 'latest'

One "no_HTTPS" condition: "SSL/TLS connection established" and this negated (Hook @ bottom of menu)
This condition type doesn't exist (anymore). I chose "Traffic is HTTP", and then NOT negated.

Issue found, not fully resolved: The order of the rules when creating the "Virtual services / Public Services" is very important.

See https://github.com/opnsense/plugins/issues/1000 & https://github.com/opnsense/plugins/issues/1925



Remaining problem:

Quote[WARNING] 087/142215 (72748) : parsing [/usr/local/etc/haproxy.conf.staging:66] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 087/142215 (72748) : parsing [/usr/local/etc/haproxy.conf.staging:68] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.

Code Select
63     # ACTION: redirect_acme_challenges                                         
64     use_backend acme_challenge_backend if acl_605e1a292ddce1.15865947           
65     # ACTION: rule_redirect_http_https                                         
66     http-request redirect scheme https code 301 if !acl_6061a61b811977.92979045 acl_605e848931a523.42844100
67     # ACTION: rule_no_host_match                                               
68     http-request deny unless acl_605e858f8de9e6.77623103 || acl_605e85820c35d5.14821142 || acl_605e8558014977.72403767 || acl_605e1a292ddce1.15    865947
69     # ERROR FILE: Unknown_FQDN                                                 
70     errorfile 403 /tmp/haproxy/errorfiles/605e84df732dd0.89785772.txt

The redirect to https (line 66) is always run before the redirect to use_backend (line 64) as shown in the logs of the acme-client

Quote2021-03-29T13:12:25   acme.sh[16275]   ] nope.nope:Verify error:Fetching https://nope.nope/.well-known/acme-challenge/{challenge key}

The full haproxy.conf.staging can be found here: https://pastebin.com/bkbU0s88[/s]
April 18, 2021, 08:10:08 PM #11
Hey there,

could you write the instructions without abbreviations? I'm going crazy right now to bring bitwarden online with a subdomain  :'(

I search and search my german interface to find out what you mean with P/HAP The rest is okay right now, although it would be more understandable with screenshots.

Thank you in advance
HC
April 27, 2021, 11:45:41 PM #12
Quote from: astromeier on February 05, 2021, 08:17:12 PM
Dear akhiltoor, thanks for this suggestion - it makes sense to revise my HowTo and add some hardcopies.
In the next month I'll switch to new hardware and V 21.1.
Then I'll rewrite the HowTo ...

I'd love to see an updated version. I'd be happy to help test things out too.

Thanks for the writeup!
May 31, 2021, 01:12:16 PM #13
I might be a new user but I have plenty of experience with this kind of stuff.
So I wrote something up in the last couple of days.

https://forum.opnsense.org/index.php?topic=23339
All of my posts are submitted with the best of knowledge and belief.


My post was helpful to you?
Feel free to click [applaud] to the left underneath my profile.
Additionally you can consider donating: https://www.buymeacoffee.com/thehellsite
May 31, 2021, 11:18:46 PM #14
WOW!
Great guide - thank you very much for sharing!
So I think I don't have to update my outdated guide - yours is more up to date and complete!
Stay secure!
Thomas

OPNsense 22.x / Qotom Q370G4 ram8G ssd256G
Print
Go Up Pages1 2
User actions