Tutorial 2024/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

[opnuser1]

Here are the requirements for the app with that install script.  Again, I tried on several servers and the only one it worked smoothly was the oracle cloud with ports 80 443 open and free.

If I am not mistaken, this is the reason for needing port 80 open:
To install the service, you first download some files.  Then you need to build it out.  Once it is built, then you go to the website on port 80.  Here there is a button to run an installation script.  Once that is finished, now the website is secured on an nginx reverse proxy.  So port 80 is initially needed to install.  You can do without, but then you can't use the handy script.

[opnuser1]

I apologize for being a moron.  I entered 0:0:0:0 instead of 0.0.0.0
You can disregard all below, I just left it for reference.  I am now checking if the servers are working as expected, etc.  But HAProxy shows no errors.

[TheHellSite]

Here are the requirements for the app with that install script.  Again, I tried on several servers and the only one it worked smoothly was the oracle cloud with ports 80 443 open and free.

If I am not mistaken, this is the reason for needing port 80 open:
To install the service, you first download some files.  Then you need to build it out.  Once it is built, then you go to the website on port 80.  Here there is a button to run an installation script.  Once that is finished, now the website is secured on an nginx reverse proxy.  So port 80 is initially needed to install.  You can do without, but then you can't use the handy script.

I think you don't quite understand the basics of networking very well.

If you place a reverse proxy, configured as described in my tutorial, in front any app this app can then run on ANY port and you will still be able to access it from external networks via haproxy on port 443/https.

The port 80 requirement might be valid. However I see absolutely no need to run one reverse proxy per app because of it. This goes against any sort of common sense. You don't need that at all. Not even during installation!

Just use haproxy and configure it exactly like in my tutorial.
Remove all other reverse proxies in your network.
Then configure all your currently active apps in haproxy.

If you now need to install another app using some install script (that for whatever reason needs port 80 during install)....
You just configure the app/service in haproxy on port 80, so you can access the install script.
After the installation is complete you simply change the port of the service in the haproxy server settings to the port the app has after installation.
Dead simple.

BTW: I don't get why you are making such a mistery out of the install scripts you are using. If you would have posted them in the beginning I could have told you right then what it is necessary.

Also remove your haproxy conf file. You exposed your domain.

[opnuser1]

Let me see if I understand...

You are recommending that I stop using my reverse proxy, which is super easy to use, and move the 40 services I have running on that and to configure them in HAproxy instead?  So, you are asking me to configure 40 services in HAproxy?  That sounds daunting to me.

If you now need to install another app using some install script (that for whatever reason needs port 80 during install)....
You just configure the app/service in haproxy on port 80, so you can access the install script.
After the installation is complete you simply change the port of the service in the haproxy server settings to the port the app has after installation.
Dead simple.

Isn't this what I've been trying to do?  What do you mean by configure the app in haproxy on port 80?  In order for it to work, the domain needs to already be pointing to this app and accessible already from the outside on port 80.  I can't do it internally.  It's a web install script.

I like your recommendation, but it is not yet completely making sense to me, sorry.  I may have to chew on it for a while.

And there will come another time where I will want port 80 forwarded to yet another server.  I just want to be able to do it without modifying all the other apps that are already running fine.  If I do what you are saying and then later I want port 80 going somewhere else, then I'm in the same boat once again.

[TheHellSite]

Isn't this what I've been trying to do?  What do you mean by configure the app in haproxy on port 80?  In order for it to work, the domain needs to already be pointing to this app and accessible already from the outside on port 80.  I can't do it internally.  It's a web install script.

Again, please just link the install script? It is a bit hard to help without them.

Apart from that. Please verify with me...
The solution I posted here for you is working correctly, right?
The traffic is beeing routed as desired using the SNI conditions (service1.domain.com, service2.domain.com, ...) to the respective backends?

[opnuser1]

(deleted)

[opnuser1]

No matter what I do, I can't seem to get activity into HAproxy.  I think I might be having this issue:

Warning

If you configure a port that is already in use, the configuration test will be successful but the start of HAProxy will fail silently. Please ensure that the used port is free - especially if the number conflicts with the web configuration of OPNsense.

But I disconnected the camera that was intercepting port 80, and it still wont work.  I must have 10-20 servers on the LAN that use port 80 443.  What am I supposed to do?  Shut them all down, and then enable haproxy?

[TheHellSite]

No matter what I do, I can't seem to get activity into HAproxy.  I think I might be having this issue:

Warning

If you configure a port that is already in use, the configuration test will be successful but the start of HAProxy will fail silently. Please ensure that the used port is free - especially if the number conflicts with the web configuration of OPNsense.

But I disconnected the camera that was intercepting port 80, and it still wont work.  I must have 10-20 servers on the LAN that use port 80 443.  What am I supposed to do?  Shut them all down, and then enable haproxy?

Yes. This is how it works. You can't have multiple services listening on the same port. Pure logic.

Again, you DON'T need all services to have direct access to port 80 + 443, that is why you are setting up this reverse proxy.
It will handle this for you based on the subdomain.

Before you continue I HIGHLY suggest you look up what a reverse proxy does and what the difference to a basic port forward is.

Also disconnecting the camera won't help you in any way. You have to delete all port forwards for port 80+443.

A reverse proxy is a type of server that sits between a client and one or more servers, forwarding client requests to those servers. When a client makes a request to a website, for example, the request is first sent to the reverse proxy server, which then forwards the request on to the appropriate web server.

The main difference between a reverse proxy and a basic port forward is that a basic port forward simply forwards traffic from one IP address and port number to another IP address and port number. A reverse proxy, on the other hand, can act as an intermediary, providing additional services such as load balancing, caching, SSL termination, and more.

A reverse proxy can serve as a single point of contact for several different servers, distributing client requests across those servers and allowing them to work together to handle the load. In contrast, a basic port forward simply redirects traffic from one destination to another, without providing any additional functionality or benefits.

Overall, a reverse proxy is a more advanced and flexible solution for managing connections between clients and servers, particularly in situations where a large number of servers need to be balanced and coordinated to handle incoming traffic efficiently.

... from chatgpt.

[opnuser1]

(disregard)

[opnuser1]

(deleted)

[TheHellSite]

I made the change I was suggesting above for the service1 condition.  I changed it from SNI matches to "contains" and used "domain1.com" rather than *.domain1.com.

Sorry, I missed that part in the uploaded haproxy config file you uploaded.
Yes, "host matches" strictly only matches the exact host specified, no additional third-level subdomains or the main domain will be matched with such a condition.
For your usecase the host contains condition is the best choice.

Now the counter shows activity in both servers as requested, and it is routing correctly from the right place.

However, it still doesn't totally work, and for the server1 subdomains, I am getting a 526 error for invalid SSL certificate.  So I believe the traffic is going to the right place, but something is missing.
Should I have two SNI frontends?  One for port 80 and another for 443?

"it still doesn't totally work" What doesn't totally work (apart from the SSL error)?

I am a little confused now.
So the routing which FQDN of each of your domains goes to which reverse proxy and then from there to the desired service. This part if working, right?


About the SSL error, this one seems to be Cloudflare specific: https://community.cloudflare.com/t/community-tip-fixing-error-526-invalid-ssl-certificates/44273
I am not using them since I don't trust them so I can't help here.

But I can tell you that the method we just implemented can only inspect and route HTTPS/TLS traffic since SNI is an extension of the TLS protocol.
So unencrypted connections (like HTTP) won't work with the rules.

Therefore you will have to either use HTTPS from external networks only OR simply follow my whole tutorial and then create these conditions (host contains "domain1", host contains "domain2", the NoSSL_condition from my tutorial) and two rules (rule1: domain1_condition + NoSSL_condition goes to domain1_backend / rule2: same like rule1). Then place these rules on the HTTP_frontend of my tutorial.
You will not need the HTTPtoHTTPS_rule since you insist on using HTTP traffic.

[opnuser1]

Ah thank you very much.
Ultimately, I think everything you instructed is working.  Even though that is a cloudflare specific error, it tells me that I probably need a different frontend for https and http, like your tutorial does.  This way you customized for me does indeed forward 80 and 443 successfully, but I need different conditions on each.

Yes, I like your suggestion to follow the original tutorial, I have been studying it and I like everything about it.  And yes, i do insist on the port 80 as well, so that would be the difference.  I was looking at it already and trying to figure out how to modify it for port 80, but you answered in the post above.

I am also keeping track of all this in a separate article, and I will post it when it is successful here to add to yours, but if all it is is not have that redirect rule, then not really a big deal I guess.  Let me try, and I'll report back.  Thanks, and it was amazing to see the traffic get routed properly.

edit...by "doesn't work" I mean the sites I am visiting externally don't load or anything.  I don't know the exact reason why, but I'm certain it has to do with what you said about SNI and TLS and there need to be different things happening in each pipe/port. 

[opnuser1]

(deleted)

[TheHellSite]

I made a diagram first to help.   Following your instructions, wouldn't I also have to create additional backends, rules, and servers since I am processing 80and 443 separately?  I included my attempt in the picture.

Not necessary, just leave the port in the server settings blank.
This way you can use a single server + backend combination (server1 + backend1) simultaneously for port 80/443/... as the port used to connect to the front will be used to connect to the server.

So in your diagram you would simply remove the port from server1+server2 and use rule1+rule2 on both the HTTP and HTTPS frontends.

Also you have a VERY big mistake in your current diagram. Unless you really want to load balance between server1 and server2 you HAVE to use a dedicated backend for each server. I think you still didn't quite understand this. Having both servers (server1+server2) in the same backend (http_backend or https_backend) will load balance between the servers and therefore send requests to different servers for each outside connection. I highly doubt you would want this, UNLESS load balancing is really the intend of your setup. But I thought you just wanted to route/forward http(s) traffic to the correct backend proxy?

You need...
server1_backend containing server1_server (port left blank)
server2_backend containing server2_server (port left blank)
server1_rule containing server1_backend as target
server2_rule containing server2_backend as target
http_frontend:80 containing server1_rule and server2_rule
https_frontend:443 containing server1_rule and server2_rule

This way no load balancing is done but the outside requests will be forwarded to the correct server:port target.


Also I thought you where already redirecting the HTTPS traffic using the SNI_frontend and steps I provided earlier?!
Sorry but your whole situation is very confusing!

[opnuser1]

Thank you.  yes it is confusing due to my lack of knowledge.  I'm going to redo the diagram and post it again based on these latest comments.

Yes, i do not intend to load balance.  You already understand that better than I do.

I didn't realize I could keep using my previous setup you helped me with.  I will continue with that.  I was about to go through your whole tutorial from scratch, and just remove that redirect rule.