Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Topics - TheHellSite

#1
This post is dedicated to the moderators of this forum.
Sadly we are unable to directly see who they are so I have to create this topic.

It has come to my attention that this subforum is beeing used by many (mostly new) users to create topics about issues they are facing with OPNsense.

From the name of this subforum I had the assumption that it is solely dedicated to sharing tutorials / workarounds on how to achieve certain setups / configure plugins and so on.
However scrolling trough the list of topics makes me wanna cry because there are a lot of totally unrelated topics.


It would be nice if the moderators would step in there and delete such posts or move them to the corresponding subforums.
An alternative solution could be to not publish / keep hidden all new post of this subforum until a moderator approves them. And if the topic is not a tutorial or FAQ then either delete them with a notice to the poster or move them to the corresponding subforum.


Kind Regards
TheHellSite
#2
Hello,

I just discovered some MTU issues with the WireGuard connection to my VPN provider.

Some insights
- OPNsense latest updates
- ISP requires PPPoE over VLAN_7 on WAN
- both WAN and LAN use IPv4 only
- local wg interface MTU=1432
- WireGuard kmod package installed
- selective routing to external VPN provider (AirVPN)

Since my WAN is using PPPoE and I am only using IPv4 I did set the local wireguard interface to MTU 1432.
Which should be totally fine: https://lists.zx2c4.com/pipermail/wireguard/2017-December/002201.html

My OPNsense connects to my external VPN provider as a "client" (peer) and routes all the traffic of my VPN-enabled_VLAN through this WireGuard tunnel.

MTU causing SSL issues
What I recently noticed is that ALL clients of the VPN-enabled_VLAN have issues with SSL connections when being routed through the WireGuard tunnel. So I started tweaking the MTU of the OPNsense WireGuard interface which didn't solve the issue.

I then stumbled across this post: https://airvpn.org/forums/topic/49877-completed-wireguard-beta-testing-available/page/4/?tab=comments#comment-170785
The author is describing the exact same issue and said a workaround is to lower the MTU of the clients to 1392. So I tested this and it worked - no more SSL issues. I then raised the MTU until the problem occurred again.

Workaround
Set OPNsense WireGuard interface MTU=1412.

Question
Is there any other way of solving this without changing the MTU of each client device / VM that is using the VPN-enabled_VLAN?




Final solution

https://airvpn.org/forums/topic/49877-completed-wireguard-beta-testing-available/page/9/?tab=comments#comment-195875

1. Set the MTU value in the WireGuard tunnel configuration.
OPNsense --> VPN --> WireGuard --> Local --> NAME_OF_WG_TUNNEL --> MTU=1420

2. Create firewall normalization rule.
https://docs.opnsense.org/manual/how-tos/wireguard-client.html#step-6a-create-normalization-rules

3. Note
I personally have to use MTU=1412 and MSS=1352 since my WAN requires PPPoE.

#3
Hello,

I have a small routing problem with my SiteToSite VPN setup.

I only need clients of Site B to access the network of Site A.
Site A also has RoadWarriors connecting to it.
Access from Site A to Site B is not necessary.

I configured WireGuard on both sites as seen below.
The Site A WireGuard RoadWarrior firewall rule allows access to any. (1__site-a-firewall-rules.png)
The Site A WireGuard SiteToSite firewall rule allows access to any. (1__site-a-firewall-rules.png)
Then I created a WG_STS_A gateway on Site B pointing to the peer address of Site A.
Then I created a firewall rule on Site B that routes requests to the Site A subnets via the WG_STS_A gateway. (3__site-b-firewall-rules.png)

Everything is working almost just fine.
Site B has access to all LAN clients of Site A except for the modem webinterface.
Clients on Site A that would like to access the modem webinterface reach it using the WAN_MODEM gateway. (2__site-a-interface-wan_modem.png)
The modem webinterface access is working fine for all local clients on Site A and for all WireGuard RoadWarrior clients connected to Site A.

However Site B clients can not access the modem webinterface. Ping is also not working.
The firewall logs indicate that Site B clients can reach the modem address (10.55.1.1) but the reponse seems to get lost.
The firewall logs don't indicate any dropped / blocked packages.

I hope someone can tell me what I am missing here.

WireGuard on Site A
===================
    RoadWarrior Config
    ==================
        Tab:            Local
        Name:           RoadWarrior
        Instance:       0
        Tunnel Address: 10.55.11.1/24
        Peers:          notebook, phone, ... many more (except Site-B)
        Disable Routes: enabled

            Peer Example
            ============
            Tab:        Endpoints
            Name:       notebook
            AllowedIPs: 10.55.11.21/32
                        (notebook peer)

    SiteToSite Config
    =================
        Tab:            Local
        Name:           SiteToSite
        Instance:       1
        Tunnel Address: 10.55.22.1/30
        Peers:          Site-B
        Disable Routes: disabled
                        (I had to disable this so Site A clients could respond to Site B requests.)
                        ((Otherwise I would have had to manually create a STS_B_Gateway and STS_B_Route in the OPNsense settings.))

            Peer Example
            ============
            Tab:        Endpoints
            Name:       Site-B
            AllowedIPs: 10.55.22.2/32, 10.136.0.0/16
                        (Site B peer), (Site B subnets)



WireGuard on Site B
===================
    SiteToSite Config
    =================
        Tab:            Local
        Name:           SiteToSite
        Instance:       1
        Tunnel Address: 10.55.22.2/32
        Peers:          Site-A
        Disable Routes: enabled

            Peer Example
            ============
            Tab:        Endpoints
            Name:       Site-A
            AllowedIPs: 10.55.0.0/16
                        (Site A subnets)
#4
Hello,

I had my road warrior WireGuard setup running on OPNsense_A at Site_A for quite some time now.
My mobile clients have access to my LAN or can even tunnel their entire traffic through my OPNsense.

At Site_B I have my OPNsense_B and SOME clients in that network need access to the services at Site_A.

So I added the OPNsense_B "sort of" as another road warrior client to the WireGuard instance of OPNsense_A.

I didn't enable the pull routes feature of the wireguard interface since I only want some clients to go through the tunnel. So I added the gateway IP to the wg_interface and created a gateway in settings of OPNsense_B.
At this point the tunnel was already working and I was able to ping the wireguard interface address at Site_A using the OPNsense diagnostics tool.

I then created a firewall rule (see attached files) that will only route selected clients to the network of Site_A and thought that would be enough. I was wrong, I also needed to add a NAT outbound rule (see attached files) that translates everything to the OPNsense_B_WG_interface address.

Can someone please explain to me why that outbound NAT rule is necessary? I know what it does but I just can't figure out why it is necessary. Without the outbound rule I can see that my OPNsense_B is routing the traffic to OPNsense_A but in the logs of OPNsense_A is no sign of it. The Site_B networks are listed in the Allowed IPs of Site_B peer.

Ultimately I would like OPNsense_A and the services in that network to know the real local IP of Site_B clients that accessed them.

(I know that the firewall rule is currently giving the whole VLAN_CLIENT network access to the network of Site_A. I will change this when everything is working as expected.)
#5
Hello,

when I started implementing HAProxy in my network I couldn't find any complete and well written guide out there. I had to puzzle everything together from various websites.
So I thought I would save many of you a lot of time and provide my ultimate HAProxy on OPNsense tutorial. :)

This tutorial will show you how to configure HAProxy as a reverse proxy on OPNsense using wildcard certificates from Let's Encrypt.
It is going to be a step-by-step guide with images on how to set things up while also explaining why we set things up in a certain way.
I will try to make this as complete and detailed as possible.
If you think that there is anything wrong or missing, feel free to tell me about it and I will consider changing it.




Consider Supporting Free Resources


If you find this tutorial helpful and it saves you a significant amount of time, please consider showing your appreciation by buying me a beer. Your contribution will be a valuable recognition of the time and effort I have put into creating this content and will help me to continue providing quality resources for others.

Thank you for your consideration,
TheHellSite




No More Free Support

Due to the increasing number of support requests I've been receiving, both directly in the topic and via DM, I regret to inform that I can no longer provide free assistance. Balancing my real job and personal life has become extremely challenging. While I genuinely want to help everyone resolve their issues to get things up and running smoothly, I find it difficult to allocate the necessary time without sacrificing my personal commitments.

In addition, it has come to my attention that some individuals seeking help are not thoroughly reading the provided tutorial or lack the fundamental knowledge of networking. This has been a recurring issue and has made the support process increasingly frustrating.

I sincerely appreciate your interest in my expertise and if you would like to receive my assistance, I am more than happy to provide you with the details via DM.

Thank you for your understanding in this matter,
TheHellSite




This configuration is tested to be working on OPNsense 24.1.x with the latest updates as of 20240207.




How To Ask For Help

When asking for help in the topic ALWAYS include the below information:

1. ALWAYS include the HAProxy Config Export.
2. ALWAYS include relevant HAProxy errors and/or log entries.
3. ALWAYS include details about your setup, your goal, the service, ... anything relevant to the issue.

Again, please note that I can no longer provide free support.




Changelog

  • 20210603
    • Fixes and some enhancements
  • 20210611
  • 20210729
    • Added an alias for the HAProxy ports and updated the WAN interface firewall rule with it.
      This leaves us with only one firewall rule instead of two and makes even more sense if one is using additional frontends on different ports.
      Thanks @_Alchemist_ for the suggestion.
    • The tutorial is now using a map file instead of "condition + rule" for service configuration.
  • 20210730
    • Added an explanation on how to configure local-access-only subdomains in HAProxy.
  • 20210801
    • Added an explanation about using self-signed certificates for internal communication to the FAQ.
  • 20211110
    • Updated the Let's Encrypt part since the service has been renamed to ACME client.
  • 20220411
    • Updated the Let's Encrypt part because of changes to the wildcard certificate generation.
  • 20220604
    • Updated the DynDNS part to use the newer "Dynamic DNS Client" (os-ddclient) plugin.
      The previous "Dynamic DNS Support" (os-dyndns) plugin will be removed with the release of OPNsense 22.7.x.
  • 20220611
    • Changed the IP address of the "SSL_server" to one that belongs to the localhost subnet.
    • Updated the "NoSSL_condition" based on the HAProxy docs.
  • 20230223
    • Updated Part 7 to avoid confusion.
  • 20230304
    • Added Part 8 to hide the certificate on access by IP.
  • 20240201
    • Updated Part 5 - OCSP updates are now built into HAProxy. No external Cron job is necessary anymore.
      Everyone please take note of the warning from @vladnik.




Current Ciphers and Cipher Suites for a 100% A+ SSLLabs rating

Last updated/verified on 20230223 using Mozilla SSL Configuration Generator.
https://ssl-config.mozilla.org/#server=haproxy&version=2.4.17&config=intermediate&openssl=1.1.1o&guideline=5.6

All ciphers with a strength of 128 bit or below have been removed in order to get a 100% A+ rating at SSL Labs.

Cipher List
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384

Cipher Suites
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256





What will the end result look like?
We will have a wildcard SSL certificate from Let's Encrypt that gets an A+ rating with a score of 100% in each category in SSLLabs SSL Server Test.
https://www.ssllabs.com/ssltest/

We will also have two levels of load balancing our services.

Level 0 - SSL Offloading disabled
WWW --> WAN interface --> OPNsense --> HAProxy SNI Frontend --> internal servers / services

Level 1 - SSL Offloading enabled
WWW --> WAN interface --> OPNsense --> HAProxy SNI Frontend --> HAProxy SSL Frontend --> internal servers / services

Visual Schematic
(Idea: @cookiemonster)





What are we going to do?

  • We will install the necessary plugins.
  • We will use a free DynDNS provider (https://desec.io/) that supports the DNS challenge which is mandatory in order to obtain a wildcard certificate.
    If you already have your own domain and your hosting provider offers an API that is supported by the Let's Encrypt (ACME Client) plugin for OPNsense, you can use it instead.
  • We will create a wildcard SSL certificate using Let's Encrypt.
  • We will configure the necessary firewall rules and change some OPNsense settings in order for HAProxy to function properly.
  • We will use HAProxy to do SNI (explanation below) and SSL offloading.
  • We will enable access to HAProxy from the internal network.




FAQ

  • Why are we using wildcard certificates and not just regular certificates?
    For me the main reason is simplicity, there is no need to set up multiple certificates for multiple subdomains.
    Also you don't need to add any rules in HAProxy or your firewall for the ACME plugin to function correctly as the DNS challenge doesn't need this.
    If you want to read more about the differences follow the links below.
    https://sectigostore.com/page/ssl-vs-wildcard-ssl-certificate/
    https://comodosslstore.com/resources/whats-the-difference-wildcard-certificate-vs-regular-ssl-certificate/

  • How does the DNS challenge work?
    https://letsencrypt.org/docs/challenge-types/

  • What is SNI?
    https://en.wikipedia.org/wiki/Server_Name_Indication
    As you already saw above we are going to do it in two stages and not just one.

  • Why are we using a virtual IP?
    You don't have to!
    I am only doing it because I don't want to use the localhost as my "SSL_server". I just prefer to keep things isolated from each other.
    You can of course simply use the localhost (127.0.0.1) as your "SSL_server" and let your "HTTP_frontend" / "HTTPS_frontend" listen to that IP.

  • Why are we doing 2-Level-SNI?
    The main reason for this is so we can load balance services that don't require additional SSL offloading, f.e. OpenVPN over TCP.
    Basically we will have two frontends listening on port 443, one with and one without SSL offloading.

  • How can we load balance TCP traffic that we don't want to get SSL offloaded, f.e. OpenVPN over TCP?
    In my tutorial I only explain how to "redirect+load balance SSL offloaded traffic".
    This is because I myself don't have (yet) the need to actually load balance any non SSL traffic.
    It is perfectly possible to do this but I won't cover it in the guide. You can hit me up if you need help with this.

  • How can I add new services in HAProxy?
    It is almost everytime the same procedure to configure / add a new service in HAProxy.
    The long way is to...

    • Create the server or servers, f.e. "NEW_server".
    • Create the corresponding backend containing "NEW_server", f.e. "NEW_backend".
    • Create the condition that matches the traffic of "NEW_server", f.e. "NEW_condition".
    • Create the rule that redirects traffic to "NEW_backend", f.e. "NEW_rule".
    • Add "NEW_rule" to your "SNI_frontend" (if you don't need SSL offloading) or to your "HTTPS_frontend" (if you need SSL offloading).

  • Is there a faster way of adding new services if I have a lot of subdomains and services?
    Yes there is! Thank you @sorano for pointing this out.
    https://www.haproxy.com/documentation/hapee/latest/configuration/map-files/overview/
    https://www.haproxy.com/documentation/hapee/latest/configuration/map-files/syntax/
    Map files are a great way of adding new services to your HAProxy configuration.
    It is most of the time much faster than creating a individual condition + rule for each service and then adding each rule to your frontend.
    To sum this up, the short way is to...

    • Create the server or servers, f.e. "NEW_server".
    • Create the corresponding backend containing "NEW_server", f.e. "NEW_backend".
    • Create a map file containing all of your subdomains (of your services) with the corresponding backend.
    • Create a rule that uses that map file.
    • Add this rule to your desired frontend.
    You might be asking yourself now: Why is this faster?
    Well, because next time you add a new service you will only need to create the server, the backend and add the combination of "subdomain to backend" to your map file.

  • Do I need to enable "SSL" in the Real Server configuration of a service?
    You need to think of a reverse proxy setup like this.

    WWW ---Stage 1---> yourdomain.tld ---Stage 2---> OPNsense + HAProxy + LE ---Stage 3---> internal services

    Stage 1 + 2
    Public facing external traffic. Traffic in these stages is now always encrypted with a verified SSL certificate. In this case it is created and verified by Let's Encrypt.

    Stage 3
    Local facing internal traffic. Traffic in this stage can or can not be encrypted, depending on your service setup. This is the traffic from HAProxy to your internal service. It doesn't need to be encrypted because you can consider your internal network as trusted.
    However it is still strongly advised to also run this traffic encrypted.
    In HAProxy you only need to check the "SSL" box in your real server setting for this.
    But then you also need to actually enable SSL encryption on that service, f.e. by installing a self-signed certificate on that service and enabling HTTPS. Even though using a self-signed certificate will give you a warning by your browser when accessing the service directly and not through the reverse proxy, the traffic is still encrypted. The certificate is just unverified, which isn't that much of an issue since we are using our reverse proxy in front of it anyway.
    How to actually do this this depends on the service but this should be covered somewhere in its manual.

    You can read more about this here: https://www.globalsign.com/en/ssl-information-center/dangers-self-signed-certificates




The Configuration

Part 1 - Plugin Installation
  • Make sure your OPNsense is up to date and that you are using the OpenSSL firmware flavour as the LibreSSL version doesn't support TLS_1.3 as of writing this.
    In your OPNsense, go to "System --> Firmware --> Updates" and install all updates.

  • Go to "System --> Firmware --> Plugins" and install the following plugins: os-acme-client, os-ddclient, os-haproxy




Part 2 - DynDNS configuration

  • Visit https://desec.io/signup, create your account and verify your email address.

  • Visit https://desec.io/domains and create your domain by clicking on the "round yellow + icon" in the top right corner.
    Your domain has to be in the form of "your_subdomain.dedyn.io". I will use "tutorial.dedyn.io" as an example.
    Note: Please don't get confused by the word "domain" here as you are not really creating a proper domain but actually just a subdomain of the domain "dedyn.io".

  • Visit https://desec.io/tokens, create a token for your domain and name it accordingly.

  • Save the token somewhere secure as we will need it twice during this tutorial and you also might need it again in the future.
    This token will allow the DynDNS service and the Let's Encrypt plugin of your OPNsense to authenticate to the API of deSEC.

  • The next step is to add a CAA record containing the information on which certification authority (CA) is allowed to issue SSL certificates for your domain.
    This is important in order to get an A+ rating from SSLLabs.
    Visit https://desec.io/domains/your_subdomain.dedyn.io and add the CAA recoard by clicking on the "round yellow + icon" in the top right corner.

  • The next step is to add a CNAME record for your services.
    If you don't need or want to do this because you only have one service, just skip this step. But in my opinion it makes sense to do this as it simplifies loadbalancing the services with HAProxy.
    You can either add an individual CNAME record per service, f.e. "plex.your_subdomain.dedyn.io", "mail.your_subdomain.dedyn.io" and so on.
    OR
    You just create a single wildcard CNAME record "*.your_subdomain.dedyn.io" for all your services.
    With the latter "any_string.your_subdomain.dedyn.io" will resolve to the IP of "your_subdomain.dedyn.io".

  • In your OPNsense go to: Services --> Dynamic DNS --> General settings
    Check that "Enable" is ticked.

  • In your OPNsense go to: Services --> Dynamic DNS --> Accounts
    Click on "Add", fill out the information accordingly, hit "Save" and then "Apply".
    (The password is your token.)

  • If everything went right "your_subdomain.dedyn.io" and "any_string.your_subdomain.dedyn.io" should resolve to your public IP.
    You can test this by going to: Interfaces --> Diagnostics --> Ping
    If it doesn't work wait a few minutes as it can take some time before all the DNS providers around the world have your hostname in their database.
    If it still doesn't work check your configuration.

  • Now that we have our DynDNS provider all set up we will want to obtain our SSL certificate.




Part 3 - Let's Encrypt (ACME Client)

  • In your OPNsense go to: Services --> ACME Client --> Settings
    Change the settings according to my image. We don't need the HAProxy integration as we are obtaining our certificates using the DNS challenge.

  • Next go to: Services --> ACME Client --> Settings --> Update Schedule
    Here we will configure at which time of the day our certificates are renewed.
    They won't be renewed everyday as the ACME client will first check if the certificates are close to expiration and if they are not they won't get renewed.
    You want this to happen at a time of the day where there is not much load on your services as the ACME plugin restarts HAProxy so it can use your new certificates which results in a very short downtime of HAProxy.
    You will also want this to NOT happen at any full hours (f.e. 3 am) because these are times when the servers of Let's Encrypt could be so busy that your certificate renewal fails.
    Change the settings according to my image.

  • Next go to: Services --> ACME Client --> Accounts
    Create a new account. You can name it whatever you like, I usually use the domain name (if it is a real domain) or in this case the FQDN.
    Note: We will want to use the staging environment "Let's Encrypt Test CA" for now. (https://letsencrypt.org/docs/staging-environment/)

  • Next go to: Services --> ACME Client --> Automations
    Create the automation to restart HAProxy after our certificates have been renewed.

  • Next go to: Services --> ACME Client --> Challenge Types
    Add the DNS challenge for deSEC.

  • Next go to: Services --> ACME Client --> Certificates
    Add the certificate for your domain according to the image below.
    I prefer to use Elliptic Curve Cryptography (ECC). (https://en.wikipedia.org/wiki/Elliptic-curve_cryptography)
    But you can of course also use RSA keys just make sure to set the key length as high as possible in order to get an A+ rating from SSLLabs.

  • Next go to: Services --> ACME Client --> Certificates
    Now we need to forcefully issue our staging certificate so we can test things out and don't have to wait for the next update schedule.
    To do this click on the button marked in the image.

  • Next go to: Services --> ACME Client --> Log Files --> ACME Log
    If your certificate is issued successful your log should look similiar and you can proceed to the next step.
    If it doesn't check your configuration.

  • Next go to: Services --> ACME Client --> Accounts
    Since we successfully issued our staging certificate we can now leave the test environment and issue our production certificate.
    To do this edit your account and change the ACME CA from "Let's Encrypt Test CA" to "Let's Encrypt [default]".

  • Next go to: Services --> ACME Client --> Certificates
    Now again forcefully issue your certificate like we did before.
    This time it you should receive a valid and trusted SSL certificate. Make sure to check the ACME log for any errors though!

  • If this went well, we can now proceed to the prepare everything for HAProxy. To do this we have to configure some things in your OPNsense.




Part 4 - System preparation

  • In your OPNsense go to: System --> Settings --> Administration
    You only need to check "Disable web GUI redirect rule" and change the "Web GUI TCP port" to a custom one.
    Otherwise HAProxy will not function correctly as you will propably want to access your services from the WWW using the default HTTPS port (TCP 443).

  • Next go to: Interfaces --> Virtual IPs --> Settings
    NOTE: This step is optional, see FAQ! You can safely skip this step and use the localhost instead for your "SSL_server".
    But if you would like to do it my way then you will need to create a virtual IP that is in a different subnet than any of your other networks.
    Preferably you would chose an IP that belongs to the localhost subnet in order to avoid IP conflicts in your local network. (https://en.wikipedia.org/wiki/Localhost)
    In this tutorial "127.4.4.3/32" is going to be the IP on which the "HTTP_frontend" and "HTTPS_frontend" will be listening on.

  • Next go to: Firewall --> Aliases
    Now we are going to create an alias for the ports that HAProxy will be listening on.
    In most setups you will probably need at least 80 and 443.

  • Next go to: Firewall --> Rules --> WAN
    Now we are going to allow any inbound traffic hitting our WAN interface on the ports specified in the "HAProxy_ports" alias.
    Create a new firewall rule with the content below.

    If you did it right it should look like this, make sure that the rule is above all other rules in the list.

  • Now we can finally configure HAProxy and make our services available on WAN.




Part 5 - HAProxy configuration

  • In your OPNsense go to: Services --> HAProxy --> Settings --> Service
    Change the settings according to the image below.

  • Next go to: Services --> HAProxy --> Settings --> Global Parameters
    Change the settings according to the image below.
    As our certificate has the OCSP Must Staple extension we need to update HAProxy's OCSP data regularly. If we don't do this clients connecting to our services will get security warnings or won't connect at all.
    Note: The number of HAProxy threads should not exceed the number of CPU threads of your OPNsense.

  • Next go to: Services --> HAProxy --> Settings --> Default Parameters
    Change the settings according to the image below.

  • Next go to: Services --> HAProxy --> Settings --> Real Servers
    First we will add our "SSL_server" using our virtual IP (or localhost IP if you wish) to do the SSL offloading.

    Then you can add all your other services according to their individual configuration.

  • Next go to: Services --> HAProxy --> Settings --> Virtual Services --> Backend Pools
    Here we first create our "SSL_backend". This is the backend to which our "SNI_frontend" sends most of its traffic to.
    As you can't mix HTTP mode and TCP mode in a frontend to backend relation, make sure that the "SSL_backend" is set to TCP mode since our "SNI_frontend" is also running in TCP mode.

    Now we create the backend that belongs to an actual service. You will need one backend for each service.
    If you have multiple servers serving the exact same content than you will want to add all servers into a single backend so HAProxy can actually balance the load between the servers.

  • Next go to: Services --> HAProxy --> Settings --> Rules & Checks --> Conditions
    Here we will only create a "NoSSL_condition", which is necessary in order to identify non-HTTPS traffic.

  • Next go to: Services --> HAProxy --> Settings --> Advanced --> Map Files
    Here we will create a new map file "PUBLIC_SUBDOMAINS_mapfile" for our public subdomains that we want to access from outside of our network.
    Please read the FAQ about Map Files first!
    This map file is telling HAProxy that any FQDN that starts with "plex" belongs to our "PLEX_backend" (which belongs to our "PLEX_server").

  • Next go to: Services --> HAProxy --> Settings --> Rules & Checks --> Rules
    Here we add the rules that decide what to do with the traffic based on our map files (or conditions if necessary).
    The "HTTPtoHTTPS_rule" upgrades the HTTP connection to HTTPS so that the client connects to the "HTTPS_frontend" instead.

    The "PUBLIC_SUBDOMAINS_rule" maps our subdomains to our backends using the map file we created in the previous step.

  • Next go to: Services --> HAProxy --> Settings --> Virtual Services --> Public Services
    Here we will create the frontends that are listening on our interface IPs and the virtual IP we created earlier.
    At first we will create our "SNI_frontend" which will decide wether the traffic is going to be SSL offloaded or not.
    You will have to place the rules for all of your services that you don't want to get SSL offloaded in here.
    Our default backend in this frontend will be the "SSL_backend" that redirects all traffic to our virtual "SSL_server" which is actually our "HTTPS_frontend".

    No we will create our "HTTP_frontend". Make sure to place the "HTTPtoHTTPS_rule" in this frontend!
    This frontend is necessary in order to redirect HTTP traffic to HTTPS. But you could also use it to serve non SSL encrypted services on port 80.

    No we will create our "HTTPS_frontend".
    This will be our primary frontend which is doing the SSL offloading using our earlier created Let's Encrypt certificate.
    You will have to place the "PUBLIC_SUBDOMAINS_rule" and all other rules of your services that you want to get SSL offloaded in here.
    Don't forget to change the "Cipher List" and "Cipher Suites" with the ones at the top of this tutorial "Current Ciphers and Cipher Suites for a 100% A+ rating at SSLLabs".

  • Done.
    Access from external networks should now already be working.
    Just try to access your URL "your_service.your_subdomain.dedyn.io" from any device that is not connected to your local network, f.e. a smartphone on cellular data.

  • You can now also verify your SSL settings: https://www.ssllabs.com/ssltest/

  • If you configured everything in HAProxy and the ACME Client according to my template you should get an A+ rating with a score of 100% in each category.

  • The last thing for us to do now is enable access from internal networks.




Part 6 - Access from internal networks
If you try to access your URL "your_service.your_subdomain.dedyn.io" from a device in your internal network, it should fail.
There are two ways of fixing this. I will cover both options but keep in mind that Split DNS (Option A) is the suggested way of doing it.
NAT Reflection (Option B) is an inferior solution since you lose the ability to track originating source IP in HAProxy when going through NAT. (@sorano)

Option A - Split DNS (https://docs.opnsense.org/manual/unbound.html#overrides)
Option B - NAT Reflection (https://docs.opnsense.org/manual/nat.html)

Option A - Split DNS (DNS Overrides)
Since you are using OPNsense you are probably also using the Unbound DNS plugin as your local DNS server.
Because of that you can easily set up DNS overrides.

  • In your OPNsense go to: Services --> Unbound DNS --> Overrides

  • Here you will need to create "Host Overrides" for each of your services. At least if you are using 2nd level subdomains "your_service.your_subdomain.dedyn.io" for you services.
    If you are running all of your services on your 1st level subdomain "your_subdomain.dedyn.io" than you will just need to override this one.

  • The IP address can be any LAN (or VLAN) interface IP of your OPNsense, I am using the LAN IP on which the "SNI_frontend" is also listening on because we set it to "0.0.0.0".

  • Access from internal networks should now be working.

Option B - NAT Reflection
Please note that NAT Reflection is only applicable on port forwarding rules so you will have to change the "Allow HAProxy" rule to a port forwarding rule with the localhost (or some random virtual IP on the localhost) as target.

  • In your OPNsense go to: Firewall --> NAT --> Port Forward

  • Here you will have to edit the "Allow HAProxy" rule we created in Part 4 - Step 3 of this tutorial.
    At the bottom of each rule there is a setting called "NAT reflection = Use system default".
    You will want to change this to "NAT reflection = Enable".

  • Access from internal networks should now be working.




Part 7 - Advanced Configuration: local-access-only subdomains
Imagine you have a service that you would like to access / protect using your brand new reverse proxy without making it available on the internet?
Well, HAProxy has got you covered!


  • In your OPNsense go to: Services --> HAProxy --> Settings --> Advanced --> Map Files
    Here you need to clone the "PUBLIC_SUBDOMAINS_mapfile", rename it to f.e. "LOCAL_SUBDOMAINS_mapfile" and add all your local-access-only subdomains along with their corresponding backends.
    Keep in mind that the content of your "PUBLIC_SUBDOMAINS_mapfile" also has to be put in the "LOCAL_SUBDOMAINS_mapfile"! I will explain why later.

  • Next go to: Services --> HAProxy --> Settings --> Rules & Checks --> Conditions
    Now you need a condition that detects if the source of the request is a local IP or a FQDN.
    You can of course also use the predefined "Source IP is local" condition.
    I am however using only specific subnets since the predefined condition is using the entire RFC1918 IP range, which I don't need!


    As I just said you can also check for a FQDN.
    But please keep in mind that HAProxy resolves those hostnames to their IPs and then checks them. But the resolving is only done once during the start / restart of HAProxy.
    So if the IP of your FQDN is changing regularly this won't work very well, except if you restart your HAProxy using a cron job like every 24 hours or so.

  • Next go to: Services --> HAProxy --> Settings --> Rules & Checks --> Rules
    Here you need to clone the "PUBLIC_SUBDOMAINS_rule", rename it to f.e. "LOCAL_SUBDOMAINS_rule", select your "LOCAL_SUBDOMAINS_SUBNETS_condition" and select your "LOCAL_SUBDOMAINS_mapfile".
    If you are also using a FQDN condition, like I do, you will need to select both your FQDN and your subnet condition together with the logical "or" operator!

  • Next go to: Services --> HAProxy --> Settings --> Virtual Services --> Public Services
    The last thing left to do is to place the "LOCAL_SUBDOMAINS_rule" before your "PUBLIC_SUBDOMAINS_rule" in your "HTTPS_frontend".

    Attention!
    Remember that I told you to also put the content of your "PUBLIC_SUBDOMAINS_mapfile" in the "LOCAL_SUBDOMAINS_mapfile"?
    This is because HAProxy is processing the rules in the frontends based on the order they appear!
    So if you place your "PUBLIC_SUBDOMAINS_rule" before your "LOCAL_SUBDOMAINS_rule" in the frontend configuration, you won't get access to your local-access-only subdomains.
    Vice versa this will also happen and you will no longer have access to your public subdomains.
    To avoid this you have to also put the content of your "PUBLIC_SUBDOMAINS_mapfile" in the "LOCAL_SUBDOMAINS_mapfile" and place their rules in the correct order.

    The correct way of placing both rules is like this.

  • Done!
    You should now still have access to your public subdomains from any network and also have access to your local-access-only subdomains from the locations you defined.




Part 8 - Advanced Configuration: Hide your certificate on access by IP
You might have noticed that if you now access your OPNsense using your public WAN IP (https://YOUR_PUBLIC_IP/) the connection will be secured and upon further inspection you will see that your Let's Encrypt certificate is beeing used.
While this is not a major security problem it still presents at least some privacy issues.
To fix this we can add the "strict-sni" parameter to the "HTTPS_frontend" so the connection will be closed when HAProxy is directly accessed using your public WAN IP (https://YOUR_PUBLIC_IP/).


  • In your OPNsense go to: Services --> HAProxy --> Settings --> Virtual Services --> Public Services
    Edit your "HTTPS_frontend" and enable the "advanced mode" in the top left corner.
    You should now be able to see the "SSL option pass-through" field in the "SSL Offloading" section, here we already added the parameter "curves secp384r1" to make use of ECC.
    Change this field from "curves secp384r1" to "curves secp384r1 strict-sni", save the changes to the "HTTPS_frontend" and apply them to HAProxy.

  • Done!
    You should now no longer get presented with your trusted Let's Encrypt certificate when accessing "https://YOUR_PUBLIC_IP/" but instead with a "SSL_ERROR_UNRECOGNIZED_NAME_ALERT".