OPNsense Forum
English Forums => Tutorials and FAQs => Topic started by: astromeier on March 20, 2019, 07:43:58 pm
-
TheHellSite made an actual V21.1 HowTo (https://forum.opnsense.org/index.php?topic=23339)
After having troubles with my first installation I removed it completely, wiped all residual data and restarted from scratch.
Here I will describe what I did:
First a short description of my hardware installation:
A cable modem is directly connected to WAN interface - the WAN-Address comes via DHCP and changes with each restart of the modem
On the LAN side there is my home network with several PC clients and a RX300S6 server hosting several VMs.
All managed via DHCP and a fixed MAC-IP address configuration.
These VMs are different servers with Homepages (NGINX/Grav) and a cloud instance (NGINX/Nextcloud).
These VM servers should be accessible from WAN via HTTPS.
A HTTP request of a FQDN should be redirected (code 301) to HTTPS address.
The HaProxy offloads the HTTPS and dispatches the requests from WAN to the corresponding server via HTTP.
My FQDNs are CNAMED to a DynDNS service, whitch points to my actual WAN address.
The Dynamic DNS plugin updates the DynDNS service when the WAN address changes.
The Let's encrypt plugin keeps an eye to the certificates for HaProxy / Offloading.
My OPNsense configuration:
- OPNsense 19.1.4-amd64 - FreeBSD 11.2-RELEASE-p9-HBSD - OpenSSL 1.0.2r 26 Feb 2019 - plain IPv4
- HTTPS port for LAN accessed Administration GUI changed from 443 to another port, listen interface LAN
- Enabled secure shell, port changed to another value, listen interface LAN & serial port enabled
- other configs @ default values
- installed plugins and chronology: 1. DynDNS, 2. HaProxy 3. Let's encrypt
- single gateway @ WAN
- Trust: No additional authorities or certificates for LE needed - they are installed by the LE plugin
- WAN config: Block private and bogon network both activated
- FW: No special rules for LAN
- FW: WAN: HTTP (P:80) and HTTPS (P:443) pass any source to any destination - default gateway
- No special NAT for HaProxy or LE plugin
- FOR the next steps i urgendly recommend for all names etc: Don't use blanks, use an underline (e.g not "my name" but "my_name"
- Plugings/HaProxy: One real server entry @ HTTP (P:80) for each VM server with a WAN accessible service and LAN internal HTTP communication
- P/HAP: Backend Pools: One Backend for each real server - no rules
- Then configure the Let's encrypt plugin: Settings: use stage environment for your first trials, check auto renewal and HaProxy integration
- create an account
- create a validation method: HaProxy HTTP integration and HTTP-01 challenge type, HaProxy HTTP frontend
- create an automation: Restart HaProxy
- create certificate entries for each FDQN: Common name is your FDQN like www.mydomain.de, Alt name e.g. mydomain.de and automation: restart HaProxy (all FDQN and Alt names must be active @ WAN!)
- Back to LE/settings: Test config and activate plugin to make the HaProxy additions by LE-plugin
- Back to Haproxy plugin:
- P/HAP: Conditions: For each FDQN a "host matches" condition
One "no_HTTPS" condition: "SSL/TLS connection established" and this negated (Hook @ bottom of menu)
One "no_ACME_challenge" condition: Copy the "find_acme_challenge" (made by LE-plugin) to a new and negate this (Hook @ bottom of menu) - P/HAP: Advanced/Error messages: Create an Error Message for request that do not fit any FDQN - Code 403, content:
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
- P/HAP: Rules: Make one rule per server/backend: If condition FDQN matches use specified Backend pool
Make one rule for HTTP to HTTPS redirection: If no_acme_challenge AND no_HTTPS do http-request-redirect with HTTP redirect "scheme https code 301"
Check if "redirect_acme_challenges" is present (made by LE plugin - Make a "no_host_match" rule: Test type Unless all of your FDQN match rules incl. the find_acme_challenge rule with an OR condition and execute function "http-request-deny"
- Virtual services / Public Services: One Frontend for HTTP (P:80) :
Listen address: 0.0.0.0:80, HTTP Frontend default backend: acme_challenge_be
Rules: no_host_match, redirect_acme_challenge, http_redirect (to HTTPS)
Error message: deny_403 if nothing matches
and one for HTTPS (P:443) but disabled since the first certificates are validated - then you can enable it.
Listen address: 0.0.0.0:443 - SSL offloading activated, HTTPS default backend one other BE of your choice
Rules: no_host_match, redirect_acme_challenge, all the redirections to your several servers,
Error message: deny_403 if nothing matches
- Go to the shell (e.g. via SSH) and check wether the path:
/var/etc/acme-client/challenges/.well-known/acme-challenge/
exists. if not create it with root:wheel owner
- Go to HaProxy plugin, test config and enable it.
- Go to LE plugin / Certifikates and (check for working on the staging platform) and Issue the certificates
- If successful, switch to productive platform and renew the certifikates
- If successful go to Haproxy / HTTPS public service / frontend and add the new certificates, enable this public service
- Restart HaProxy
Hopefully I've nothing forgotten!!!
If so - your reverse Proxy should work now and is secured by LE certificates....
Please add your infos and suggestions in this thread!
If my may is way too complicated pleas show your easy way - I'm keen on it!
stay secure!
Thomas
-
Just wanted to reply thanking you for this, it's working great! I'm home-bound for a few weeks following spinal surgery, so I'm finally getting to spend some time on my home lab. Have been trying to get this working off and on for about a month now, but after completely blowing away my previous HAProxy config and following along here step by step, it was practically easy!
I only had a couple issues following along that I wanted to share with anyone else using this guide:
First was that I got a Test Config failure after creating the first front-end. Turns out that I had to add an HTML header to the <html> tag in the Advanced/Error Message/403_deny. Don't know if this is a new requirement or what, but it's a simple fix. Altogether, it looked like this:
HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/plain
Retry-After: 60
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
Second, I wasn't exactly sure where to add the certs to the frontend (on the very last step). Turns out I got it on the first try by adding my new certs to SSL Offloading/Certificates. I did not need to make any changes to the Client Certificate Auth section (default=disabled) which is what confused me.
Again, thank you for the write-up! Nextcloud seems to be working great behind the reverse proxy, so it's on to Collabora and a whole lot of others!
-
This has been extremely useful to get Next Cloud setup using LE and HAproxy.
I did want to make one change to the HAproxy error message to have it render properly.
HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html
Retry-After: 60
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
-
Hello,
I am French so sorry if my English is not completely correct.
Following a post I saw on the French community, I think it might be good to image this tutorial with screenshots.
Why ? Because it is easier to understand on screenshots than on plain text for those who do not speak English well.
I'm from PfSense, and I find OPNsense so much more enjoyable to use. Unfortunately it is not possible to find good tutorials, like for example HAProxy / Lets Encrypt.
You are under no obligation to answer my request, but I think it could be interesting that communities can work together.
Thanks for reading this message :D
-
Dear akhiltoor, thanks for this suggestion - it makes sense to revise my HowTo and add some hardcopies.
In the next month I'll switch to new hardware and V 21.1.
Then I'll rewrite the HowTo ...
-
Hello astromeier,
Thank you very much for answering me.
It's really great, I can't wait to see the result.
I wish you good luck for the writing and the formatting of this next HowTo
Have a nice week.
-
Thank you for a great guide.
I have been set up, but I do have a question:
- Is it possible to create a wildcard certificate, and if so, how do I set this up in ACME?
-
Hi Re4mstr,
I've used the "Alt names" entry in the Certificates Options:
See attached screenshot:
Base Certificate is www.m****e.de and "Alt Names" is m****e.de for subdomains.
hth
-
Hi Re4mstr,
I've used the "Alt names" entry in the Certificates Options:
See attached screenshot:
Base Certificate is www.m****e.de and "Alt Names" is m****e.de for subdomains.
hth
Wouldn't having "domain.com" in base be exactly the same thing?
Currently I'm having subdomains in the alt-names. It kinda works, but more work than I feel is necessary.
-
Wouldn't having "domain.com" in base be exactly the same thing?
Hm, not sure - I haven't tested that....
-
Currently working on getting this set up using opnsense 21.1.3_3, os-haproxy 3.0 and os-acme-client 2.4
Aka, I'm running 'latest'
One "no_HTTPS" condition: "SSL/TLS connection established" and this negated (Hook @ bottom of menu)
This condition type doesn't exist (anymore). I chose "Traffic is HTTP", and then NOT negated.
Issue found, not fully resolved: The order of the rules when creating the "Virtual services / Public Services" is very important.
See https://github.com/opnsense/plugins/issues/1000 & https://github.com/opnsense/plugins/issues/1925
Remaining problem:
[WARNING] 087/142215 (72748) : parsing [/usr/local/etc/haproxy.conf.staging:66] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 087/142215 (72748) : parsing [/usr/local/etc/haproxy.conf.staging:68] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
63 # ACTION: redirect_acme_challenges
64 use_backend acme_challenge_backend if acl_605e1a292ddce1.15865947
65 # ACTION: rule_redirect_http_https
66 http-request redirect scheme https code 301 if !acl_6061a61b811977.92979045 acl_605e848931a523.42844100
67 # ACTION: rule_no_host_match
68 http-request deny unless acl_605e858f8de9e6.77623103 || acl_605e85820c35d5.14821142 || acl_605e8558014977.72403767 || acl_605e1a292ddce1.15 865947
69 # ERROR FILE: Unknown_FQDN
70 errorfile 403 /tmp/haproxy/errorfiles/605e84df732dd0.89785772.txt
The redirect to https (line 66) is always run before the redirect to use_backend (line 64) as shown in the logs of the acme-client
2021-03-29T13:12:25 acme.sh[16275] ] nope.nope:Verify error:Fetching https://nope.nope/.well-known/acme-challenge/{challenge key}
The full haproxy.conf.staging can be found here: https://pastebin.com/bkbU0s88[/s]
-
Hey there,
could you write the instructions without abbreviations? I'm going crazy right now to bring bitwarden online with a subdomain :'(
I search and search my german interface to find out what you mean with P/HAP The rest is okay right now, although it would be more understandable with screenshots.
Thank you in advance
HC
-
Dear akhiltoor, thanks for this suggestion - it makes sense to revise my HowTo and add some hardcopies.
In the next month I'll switch to new hardware and V 21.1.
Then I'll rewrite the HowTo ...
I'd love to see an updated version. I'd be happy to help test things out too.
Thanks for the writeup!
-
I might be a new user but I have plenty of experience with this kind of stuff.
So I wrote something up in the last couple of days.
https://forum.opnsense.org/index.php?topic=23339
-
WOW!
Great guide - thank you very much for sharing!
So I think I don't have to update my outdated guide - yours is more up to date and complete!
-
Thanks ;D
Maybe you could even point people towards it in you first post, so more people find it. :)
-
... done ... :)
-
Thanks. :)