Today suddenly certificate doesn't work. I get unasigned certificate when visit website,
When I check logs I have this error in the link. What can be reason for certificate doesn't update, because other web site is updated and it works. And there was no change on server or opensense firewall or domain settings. After update it today nothing happens, still doesn't work.
https://privatebin.povej.net/?b269015f39990e05#4KCy88sxcFxAQ7aMDBe3RV1m7eixgV5q7YPw2dSrJ6GA (https://privatebin.povej.net/?b269015f39990e05#4KCy88sxcFxAQ7aMDBe3RV1m7eixgV5q7YPw2dSrJ6GA)
Name
OPNsense.localdomain
Versions
OPNsense 24.7.3_1-amd64
FreeBSD 14.1-RELEASE-p3
OpenSSL 3.0.14
Uptime
01:57:49
Load average
0.28, 0.30, 0.26
Current date/time
Fri Sep 6 10:18:15 CEST 2024
Last configuration change
Fri Sep 6 8:55:24 CEST 2024
We've had ACME + LE randomly failing for ~ past week for no reason, not even using OPNsense for that. It eventually renews if you keep trying.
There's something broken on their end, really.
Probably related: https://community.letsencrypt.org/t/unexpected-renewal-failures-since-april-2024-please-read-this/216830
Thank you for notification.
So there is nothing to do to solve this but wait?
I can't wait and have my website not working.
I hope there is some solution for this
I already did this but didn't solve my problem
_acme-challenge.<YOUR_DOMAIN>
Well, normally you renew ACME certificates well in advance, not one day before they expire. As said, making repeated attempts worked here.
Quote
I already did this but didn't solve my problem
_acme-challenge.<YOUR_DOMAIN>
Not sure what you did where really. DNS-01 does not work the way you imagine. The client needs to create the TXT records from the token provided by ACME, dynamically via API with your DNS provider.
I have added in my DNS provider that _acme-challenge.<YOUR_DOMAIN>
probably that is not all what I need to do.
I tried to renew certificate many times but still doesn't work.
But your are not using DNS-01 at all... why would you be adding that? Plus again, those CNAME records are used for delegation, not validation. https://www.eff.org/deeplinks/2018/02/technical-deep-dive-securing-automation-acme-dns-challenge-validation
2024-09-06T08:55:24 opnsense AcmeClient: domain validation failed (http01)
The logs show that the verification ist not attempted via DNS-01, but via HTTP-01.
For that to work, you must present the ACME verification token in "http://yourtop.news/.well-known/acme-challenge/<file>".
However, your web server obviously is configured to reply with a 301 redirect to all requests on port 80 to use HTTPS:
wget -O- --no-check-certificate http://yourtop.news/.well-known/acme-challenge/x
--2024-09-06 13:05:43-- http://yourtop.news/.well-known/acme-challenge/x
Resolving yourtop.news (yourtop.news)... 31.15.171.21
Connecting to yourtop.news (yourtop.news)|31.15.171.21|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://yourtop.news/.well-known/acme-challenge/x [following]
--2024-09-06 13:05:43-- https://yourtop.news/.well-known/acme-challenge/x
Connecting to yourtop.news (yourtop.news)|31.15.171.21|:443... connected.
WARNING: cannot verify yourtop.news's certificate, issued by 'CN=R11,O=Let's Encrypt,C=US':
Issued certificate has expired.
HTTP request sent, awaiting response... 404 Not Found
2024-09-06 13:05:43 ERROR 404: Not Found.
So maybe you forgot to handle ".well-known" differently? Of course that depends on how or where you do the verification. Sometimes, the port is only handled during the verification step such that the output above is not representative. However, the timinig in you log suggests that the web server does answer, but incorrectly.
Quote from: meyergru on September 06, 2024, 01:15:23 PM
However, your web server obviously is configured to reply with a 301 redirect to all requests on port 80 to use HTTPS:
Did not even check that, since the OP claims:
Quote
there was no change on server or opensense firewall or domain settings. After update it today nothing happens, still doesn't work.
If that was the case, it'd have never worked in the first place. Sigh.
Even if you have instructed your provider to delegate something for DNS-01 challenge, this sure does not work either:
#nslookup -query=any _acme-challenge.yourtop.news
;; Got SERVFAIL reply from 127.0.0.53
Server: 127.0.0.53
Address: 127.0.0.53#53
** server can't find _acme-challenge.yourtop.news: SERVFAIL
thank you for your answers.
I am now totally lost here. I didn't setup this firewall someone else did and I am trying now to solve this problem.
I really don't know what to setup and where to solve this.
Port 80 should redirect to port 443.
Quote from: meyergru on September 06, 2024, 01:27:59 PM
Even if you have instructed your provider to delegate something for DNS-01 challenge, this sure does not work either:
#nslookup -query=any _acme-challenge.yourtop.news
;; Got SERVFAIL reply from 127.0.0.53
Server: 127.0.0.53
Address: 127.0.0.53#53
** server can't find _acme-challenge.yourtop.news: SERVFAIL
this is what I did
For that last time:
Your ACME is NOT set up to use DNS-01 so whatever you do in DNS with _acme-challenge.yourtop.news is irrelevant. (And - as also already noted, delegation is done via CNAME, not TXT. TXT is created dynamically via API, you CANNOT prepopulate it manually.)
For HTTP-01 to work, you MUST NOT be redirecting the well-known URL to HTTPS.
I know that my ACME is not setup.
So what I need to do, change my ACME to use DNS-01 or there is another way to solve this.
If not than how I need to change my ACME to use DNS-01
I have 2 domains in and first one works OK, this second one doesn't and both are set the same.
You must disable the HTTPS redirect as already noted. Then it will work. You can re-enable after you have your certificate. Forget about DNS-01 at the moment, you clearly need to do some reading on how the thing works.
Quote from: doktornotor on September 06, 2024, 02:01:20 PM
You must disable the HTTPS redirect as already noted. Then it will work. You can re-enable after you have your certificate. Forget about DNS-01 at the moment, you clearly need to do some reading on how the thing works.
I know I need to read a lot more about how it works because I don't know.
thank you for advice I will do
Quote from: doktornotor on September 06, 2024, 01:39:56 PM
For that last time:
Your ACME is NOT set up to use DNS-01 so whatever you do in DNS with _acme-challenge.yourtop.news is irrelevant. (And - as also already noted, delegation is done via CNAME, not TXT. TXT is created dynamically via API, you CANNOT prepopulate it manually.)
And for this to work, the _acme-challenge.yourtop.news zone is usually delegated because for it to work, you must be able to change it dynamically. But as already noted, you do not use DNS-01 anyway.
Quote from: doktornotor on September 06, 2024, 01:39:56 PM
For HTTP-01 to work, you MUST NOT be redirecting the well-known URL to HTTPS.
Exactly. It will only be queried via HTTP, not HTTPS (obviously, otherwise the first verification would never work, since you do not have a certificate then).
Not helping the OP issue, which is another story, but:
Quote from: doktornotor on September 06, 2024, 01:39:56 PM
For HTTP-01 to work, you MUST NOT be redirecting the well-known URL to HTTPS.
Instead of a "MUST NOT" LE itself talks about "SHOULD"
Our recommendation is that all servers meant for general web use should offer both HTTP on port 80 and HTTPS on port 443. They should also send redirects for all port 80 requests, and possibly an HSTS header (on port 443 requests).* https://letsencrypt.org/docs/allow-port-80/
Quote
Exactly. It will only be queried via HTTP, not HTTPS (obviously, otherwise the first verification would never work, since you do not have a certificate then).
The nice thing with LE is that they don't care what you provide as certificate (invalid, self-signed, etc) when doing the challenge. So even if you refreshing your cert 1 day too late, your expired certificate will be used for the refresh (ie, not checks or validation on the cert, only on a valid ACME challenge.
Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to "http:" or "https:", and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way).* https://letsencrypt.org/docs/challenge-types/
Yeah, it is not helping since that clearly does not work. There are other ACME mechanisms that work. HTTP-01 is not one of them.
Quote from: doktornotor on September 06, 2024, 08:50:34 PM
Yeah, it is not helping since that clearly does not work. There are other ACME mechanisms that work. HTTP-01 is not one of them.
DNS-01 is prefered indeed for a lot of reasons, HTTP-01 at scale is a disaster, but it does work perfectly fine over https tcp/443. In 99.9% of failures the redirection configuration is not done right (mostly NGINX), but that's Layer 8, not ACME or LetsEncrypt.
Quote from: Blisk on September 06, 2024, 02:03:07 PM
Quote from: doktornotor on September 06, 2024, 02:01:20 PM
You must disable the HTTPS redirect as already noted. Then it will work. You can re-enable after you have your certificate. Forget about DNS-01 at the moment, you clearly need to do some reading on how the thing works.
I know I need to read a lot more about how it works because I don't know.
thank you for advice I will do
Is there a simple way to disable redirection to https?
If not I need to delete half of haproxy for 2 domains.
I also tried DNS-01 challenge but there is a list of DNS service, which one to use? Some require API or username and password.
I had the ACME + LE also fail on me last weekend, on a setup that has been working for years. What worked for me was to switch to the LE test CA, force issue new certificates, switch back to the production environment, force issue new certificates again, and then it worked.
It may be a coincidence though.
As I noted originally, there definitely were upstream issues with certificate renewal recently. Encountered on multiple sites and many of them with zero OPNsense involvement.
As for the rest, I have a strong feeling that this "certificate for everyone" progress seems to be rather harmful. Apparently people having no clue whatsoever about what they are doing can now have their own trusted certs with 2 clicks or so. Until it breaks.