Tutorial 2024/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

[jonf]

I'm having problems with my certificate not renewing, automatically and manually. I get the error 'domain validation failed (dns01)' in the 'System log' tab under ACME. I haven't changed my DNS hostname and it can still be accessed from the web (albeit with the 'NET::ERR_CERT_DATE_INVALID' error), including from the SSL Labs server certificate test website. I haven't made any recent changes to my HAProxy config either.

I thought maybe my internal DNS was a problem since I have query forwarding enabled in Unbound that redirects to the DNSCrypt-Proxy app in OPNsense, so I tried disabling it so that my ISP's DNS is used instead and the same thing happened. I also tried resetting the ACME client under Settings and again the same thing happened when I tried to manually renew.

What else should I try, or what other info do I need to give for troubleshooting?

[Grenen]

Just a headsup for everyone posting here. The author updated the post with the following:

No More Free Support

Due to the increasing number of support requests I've been receiving, both directly in the topic and via DM, I regret to inform that I can no longer provide free assistance. Balancing my real job and personal life has become extremely challenging. While I genuinely want to help everyone resolve their issues to get things up and running smoothly, I find it difficult to allocate the necessary time without sacrificing my personal commitments.

In addition, it has come to my attention that some individuals seeking help are not thoroughly reading the provided tutorial or lack the fundamental knowledge of networking. This has been a recurring issue and has made the support process increasingly frustrating.

I sincerely appreciate your interest in my expertise and if you would like to receive my assistance, I am more than happy to provide you with the details via DM.

Thank you for your understanding in this matter,
TheHellSite

But perhaps someone else have a solution to my problem. I have HAProxy up and running for a few months, was working fine. In may i added local domains map file for a site. Now i deleted the map file and removed all the local domain map file rules etc. But now my public domains aren't available from my internal network anymore (they work from external access).

Ive gone through the setup and everything seems fine, and i havent changed anything in the domain override in Unbound.

https://ibb.co/vkGLPGF

Any suggestions where the confligt might be located? What else block internal access to my public domains?

[jonf]

I'm having problems with my certificate not renewing, automatically and manually. I get the error 'domain validation failed (dns01)' in the 'System log' tab under ACME. I haven't changed my DNS hostname and it can still be accessed from the web (albeit with the 'NET::ERR_CERT_DATE_INVALID' error), including from the SSL Labs server certificate test website. I haven't made any recent changes to my HAProxy config either.

I thought maybe my internal DNS was a problem since I have query forwarding enabled in Unbound that redirects to the DNSCrypt-Proxy app in OPNsense, so I tried disabling it so that my ISP's DNS is used instead and the same thing happened. I also tried resetting the ACME client under Settings and again the same thing happened when I tried to manually renew.

What else should I try, or what other info do I need to give for troubleshooting?

Got it working by changing the API key for my desec hostname. Not sure why the old one stopped working, but just in case anyone else is in the same boat as me try deleting your API key, create a new one, then paste that into the challenge type settings.

[opnuser1]

TheHellSite has provided a great, extremely handy tutorial here, so thank you for that.  Very much appreciated.

He does get annoyed when people don't know what they are talking about.  But at the same time, if someone knows all these things they wouldn't be here for help.  So I don't get that.  But it does suck up your time, so either way I get it.

[cookiemonster]

Not trying to speak for the man but my humble view is different. That is that he doesn't get annoyed when people don't know what they're talking about or ask for assistance except when they think they can just do their own setup, different to his tutorial and asking why is not working. As if it was generic haproxy help thread.
He has made the point several times that he'd help to get it working _as per tutorial_, and people have continued deviating from it and coming to this thread for help.

[TheHellSite]

Actually everything is said in my statement. But I am happy to provide an answer.

TheHellSite has provided a great, extremely handy tutorial here, so thank you for that.  Very much appreciated.

He does get annoyed when people don't know what they are talking about.  But at the same time, if someone knows all these things they wouldn't be here for help.  So I don't get that.  But it does suck up your time, so either way I get it.

Basically, what you are saying is this:
Someone makes a tutorial, people follow it, have issues with it (some because they lack BASIC knowledge of networking), request help from the creator, ...
In your logic it is now the obligation of the tutorial creator to happily provide FREE support in his FREE TIME for everyone using the tutorial?  ??? Nonsense...

(I don't mean any offense to you)

While actually the real issue here (which seems to be rising) is that nowadays many people expect that everything will get done for them by someone else! Hell, hopefully even for free.
"Issue XYZ appears, what do I do now? The tutorial doesn't cover it... Mhhh, I will just ask this random guy I took the tutorial/information from, he will know it for sure."
They even expect this in the concept of homelabbing, selfhosting, ...

Where has the effort gone to learn things on your own? To solve your own issues?
This is what drives me nuts! In todays world with the free availability of AI tools it is really not hard at all to learn new topics or solve issues on your own.

I am not a doctor, so I would never just grab some book/tutorial that explains to me how to remove a kidney, go to a related person and try it, when I don't even know how to hold a scalpel.
The same applies here! If you don't know the basics then maybe this is way beyond your CURRENT skills.

(with "you" I don't mean you in particular)

Not trying to speak for the man but my humble view is different. That is that he doesn't get annoyed when people don't know what they're talking about or ask for assistance except when they think they can just do their own setup, different to his tutorial and asking why is not working. As if it was generic haproxy help thread.
He has made the point several times that he'd help to get it working _as per tutorial_, and people have continued deviating from it and coming to this thread for help.

Someone understood the issue pretty well. :)
But to be fair, I did/do get annoyed when people don't know the basics. And IMO I have all the right in the world to do so since it is not my job to teach them these basics. Especially not if it is just a 5s online search away.
THEY are using a product/software/..., THEY want to achieve something with it, so THEY have to look around how to get it working. Plain simple.

[cookiemonster]

I'd call it fair. According to the header, the thread has been read 171056 times as of now. 37 pages of assistance.
Thank you.

[opnuser1]

I'd call it fair. According to the header, the thread has been read 171056 times as of now. 37 pages of assistance.
Thank you.

Totally fair, and above and beyond.  Helped me solve a long standing goal of mine.  I was thinking of starting a similar thread, but maybe not a good idea if i do not even know the basics.

[blackwing]

is it possible to use 1 Public IP for the Public Service that will be use by different sub domains with port 80 as its port.

    sub1.domain.com Real Server  172.16.100.20 Port 80
    sub2.domain.com Real Server  172.16.100.21 Port 80
    sub3.domain.com Real Server  172.16.100.22 Port 80
with the condition prefix base on the subdomain

Public Service has the public IP 443 and 80

I was actually trying this setup but it end up loading the same content on all subs.

[sorano]

is it possible to use 1 Public IP for the Public Service that will be use by different sub domains with port 80 as its port.

    sub1.domain.com Real Server  172.16.100.20 Port 80
    sub2.domain.com Real Server  172.16.100.21 Port 80
    sub3.domain.com Real Server  172.16.100.22 Port 80
with the condition prefix base on the subdomain

Public Service has the public IP 443 and 80

I was actually trying this setup but it end up loading the same content on all subs.

Yes it is possible.

[Twitchiz]

Is nslookup on a local PC returning the proper IPs?

Just a headsup for everyone posting here. The author updated the post with the following:

No More Free Support

Due to the increasing number of support requests I've been receiving, both directly in the topic and via DM, I regret to inform that I can no longer provide free assistance. Balancing my real job and personal life has become extremely challenging. While I genuinely want to help everyone resolve their issues to get things up and running smoothly, I find it difficult to allocate the necessary time without sacrificing my personal commitments.

In addition, it has come to my attention that some individuals seeking help are not thoroughly reading the provided tutorial or lack the fundamental knowledge of networking. This has been a recurring issue and has made the support process increasingly frustrating.

I sincerely appreciate your interest in my expertise and if you would like to receive my assistance, I am more than happy to provide you with the details via DM.

Thank you for your understanding in this matter,
TheHellSite

But perhaps someone else have a solution to my problem. I have HAProxy up and running for a few months, was working fine. In may i added local domains map file for a site. Now i deleted the map file and removed all the local domain map file rules etc. But now my public domains aren't available from my internal network anymore (they work from external access).

Ive gone through the setup and everything seems fine, and i havent changed anything in the domain override in Unbound.

https://ibb.co/vkGLPGF

Any suggestions where the confligt might be located? What else block internal access to my public domains?

[tokar86a]

delete

[blackwing]

is it possible to use 1 Public IP for the Public Service that will be use by different sub domains with port 80 as its port.

    sub1.domain.com Real Server  172.16.100.20 Port 80
    sub2.domain.com Real Server  172.16.100.21 Port 80
    sub3.domain.com Real Server  172.16.100.22 Port 80
with the condition prefix base on the subdomain

Public Service has the public IP 443 and 80

I was actually trying this setup but it end up loading the same content on all subs.

Yes it is possible.

I would love to learn how to do it. cause I've been stuck with the content of my other vm that should be on another subdomain and showing up on the other subdomain.

[sorano]

I would love to learn how to do it. cause I've been stuck with the content of my other vm that should be on another subdomain and showing up on the other subdomain.

Great!

Motivation is usually all you need so what is stopping you from learning it?

[chaoticmaster]

In case anyone runs in to this issue in part 7 of the tutorial:

You can of course also use the predefined "Source IP is local" condition.

This did not wok for me and was giving me 503 errors I assume because it was using the wrong mapfile.

I defined my subnets and it worked as expected.

192.168.3.0/24, 10.168.5.0/24