I've seen and read many posts about issues with Cloudflare, but have been using it without issue for about 1-2 years, using the generated API keys from CF. I use a wildcard domain and all renewals worked from 2022 until about 70 days ago. Then, mysteriously, they stopped working with the errors below. Hoping someone has some ideas on this as I've been beating my head against it for days.
Issue:
- Starting about 70 days ago, the renewals began failing with "invalid domain" and "Error add txt for domain"
- In the past, others have fixed this with updates (I'm current on both OPNsense and plugins) or new API keys (tried that)
- Rebuilt all stages of the cert and issue persists
- Tried with a single subdomain and issue persists
Tested:
- Recreated the verification challenge, as that's where it's failing. Same errors.
- Verified/recreated the API key permissions in case something changed on CF's end. Same errors.
- Switched to a single subdomain, rather than wildcard. Same errors.
- Recreated all stages of the request. Same errors.
- Created a new API key with correct permissions. Same errors.
- Contacted CloudFlare. They blame OPNsense, because of course they do.
- NOTE: The API key does have zone read and dns edit permissions
See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
Please add '--debug' or '--log' to check more details.
Error add txt for domain:_acme-challenge.somedomain.com
invalid domain
Adding txt value: <somestring> for domain: _acme-challenge.somedomain.com
Getting webroot for domain='*.somedomain.com'
Getting domain auth token for each domain
Single domain='*.somedomain.com'
Using CA: https://acme-v02.api.letsencrypt.org/directory
As sanity check you could try getting the wildcard cert from cloudflare from the plugin in my signature. It has the cloudflare DNS Provider and DNS-01 challenge build in. It uses libdns and this provider https://github.com/caddy-dns/cloudflare
I really don't want to learn Caddy to fix an issue that just cropped up with the built-in system. I'll consider that a last resort.
Side-note...tested again using the global API key. Also says the domain is invalid.
Lacking other options, I did try the Caddy plugin. No luck...but different results.
Example, it's setup with some.sitename.com points to handler 192.168.0.1, port 1111. I go to some.sitename.com:443 and it gives me a secure blank page. It does not forward to 192.168.0.1:1111 at all.
Progress, maybe? Still would love to know why the built-in plugin isn't working, but no one seems to want to talk about it, judging by the other threads about this. :)
Well I guess that means it is possible for you to get Let's Encrypt Certificates with TXT Records of Cloudflare. Right? So that means your API Token and the API of Cloudflare works as expected, and the issue has to be somewhere with the ACME Plugin implementation of it?
Does seem to be the case! I definitely didn't mean to break the acme plugin. :D
If you have logs of the ACME plugin, you could open an issue on github, maybe theres a fix for it upstream that can be implemented? https://github.com/opnsense/plugins
Sadly I dont know much about how the ACME Plugin works.
I noticed that when creating the cloudflare api token, Acme required:
Zone Resources set: Include | All zones. This appears to be the problem.
To sum it up:
Zone | DNS | Edit
Zone Resources | Include | All Zones
Client IP (not using this field)
TTL | set a valid date range
This appears to work OK.
Quote from: rdunkle84 on March 12, 2024, 05:06:46 PM
I noticed that when creating the cloudflare api token, Acme required:
Zone Resources set: Include | All zones. This appears to be the problem.
To sum it up:
Zone | DNS | Edit
Zone Resources | Include | All Zones
Client IP (not using this field)
TTL | set a valid date range
This appears to work OK.
Tried this. Still says the domain is invalid. I've got all zones allowed and a TTL, as well as the edit permissions.
I´m using cloudflare too.
After the latest update OPNsense 24.1.4 i get a validation failed error.
did you find a way to solve problem?
Im still having this issue on latest release (24.1.7)
I am using 24.1.6, and the Acme plugin with CloudFlare DNS-01 challenge. My certificates are updating as expected and my last certificate updated on May 12. I am using Let's Encrypt as my Acme CA, a restricted API token (zone read, DNS edit) and named certs.
Same problem here, one of my website's cert has expired now!! No clue how to fix and customer already complaining. Running
AcmeClient: domain validation failed (dns01)
acme.ch seems to have problems adding the txt, but i can't see why..
[Wed May 29 12:54:39 CEST 2024] Add txt record error.
This is geeting urgent!
Some more logs...
2024-05-29T14:56:40 opnsense AcmeClient: running acme.sh command: /usr/local/sbin/acme.sh --issue --syslog 8 --debug 2 --server 'letsencrypt' --dns 'dns_cf' --dnssleep '300' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/62b86c7fd6ddb9.24403730' --certpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/cert.pem' --keypath '/var/etc/acme-client/keys/62b86c7fd6ddb9.24403730/private.key' --capath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/chain.pem' --fullchainpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/fullchain.pem' --domain 'mydomain.com' --domain 'mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/5f806aef5d0241.03202364_prod/account.conf'
2024-05-29T14:56:40 opnsense AcmeClient: using challenge type: Cloudflare DNS Validation
2024-05-29T14:56:40 opnsense AcmeClient: account is registered: avbs-acme
2024-05-29T14:56:40 opnsense AcmeClient: using CA: letsencrypt
2024-05-29T14:56:40 opnsense AcmeClient: issue certificate: mydomain.com
2024-05-29T14:56:40 opnsense AcmeClient: certificate must be issued/renewed: mydomain.com
2024-05-29T12:54:44 opnsense AcmeClient: validation for certificate failed: mydomain.com
2024-05-29T12:54:44 opnsense AcmeClient: domain validation failed (dns01)
2024-05-29T12:54:44 opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command returned exit code '1': '/usr/local/sbin/acme.sh --issue --syslog 7 --debug --server 'letsencrypt_test' --dns 'dns_cf' --dnssleep '300' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/62b86c7fd6ddb9.24403730' --certpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/cert.pem' --keypath '/var/etc/acme-client/keys/62b86c7fd6ddb9.24403730/private.key' --capath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/chain.pem' --fullchainpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/fullchain.pem' --domain 'mydomain.com' --domain 'mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/5f806aef5d0241.03202364_stg/account.conf''
2024-05-29T12:54:29 opnsense AcmeClient: running acme.sh command: /usr/local/sbin/acme.sh --issue --syslog 7 --debug --server 'letsencrypt_test' --dns 'dns_cf' --dnssleep '300' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/62b86c7fd6ddb9.24403730' --certpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/cert.pem' --keypath '/var/etc/acme-client/keys/62b86c7fd6ddb9.24403730/private.key' --capath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/chain.pem' --fullchainpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/fullchain.pem' --domain 'mydomain.com' --domain 'mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/5f806aef5d0241.03202364_stg/account.conf'
2024-05-29T12:54:29 opnsense AcmeClient: using challenge type: Cloudflare DNS Validation
2024-05-29T12:54:29 opnsense AcmeClient: account is registered: avbs-acme
2024-05-29T12:54:29 opnsense AcmeClient: using CA: letsencrypt_test
2024-05-29T12:54:29 opnsense AcmeClient: issue certificate: mydomain.com
2024-05-29T12:54:29 opnsense AcmeClient: certificate must be issued/renewed: mydomain.com
If its a customer who is complaining, why not just buy a certificate? Getting a wildcard certificate for the domain/s fixes the problem instantly and it doesn't cost much for a business.
Quote from: Monviech on May 29, 2024, 04:08:39 PM
If its a customer who is complaining, why not just buy a certificate? Getting a wildcard certificate for the domain/s fixes the problem instantly and it doesn't cost much for a business.
Maintenance every year?
Maintenance should be a plus if you have customers since you can charge for that. :D
Quote from: Monviech on May 29, 2024, 04:08:39 PM
If its a customer who is complaining, why not just buy a certificate? Getting a wildcard certificate for the domain/s fixes the problem instantly and it doesn't cost much for a business.
Agree, but i would like to fix THIS problem. It was working for years now, something seems to be changed.
Well did you check if a new TXT record for the ACME challenge is created in the DNS?
If not something might be up with the API key. I think ive read a while ago that cloudflare refuses global API keys that can access all resources, and demand a stricter one now, but unsure.
(Hint: if you think its the api key or some other weird issue, the os-caddy plugin also has cloudflare built in. Just create a domain in it and put the api key into the dns provider, and check the dns-01 challenge. Then check the logfile if it works there. If it does, then you might have some quirks with the acme sh script of the acme plugin in some way.)
I tried with ZoneID Key already, same result. I can't see any TXT records but ACME plugin normally removes it after validation. Maybe im too slow to catch it. I also tried to add the key manually, but on every round ACME generates a new key.
It sounds like you are using a global API key. See here:
https://developers.cloudflare.com/fundamentals/api/get-started/
Try creating an API token with the correct permissions and use that for the challenge.
Not sure if it helps, but noticed a behavior change at some point. If you have a double entry in your cert with the common name present as a SAN name, it will fail at the txt stage the second time it tries to validate.
Had the same issue on a wildcard cert. Solved it by removing the SAN entry.
The SAN value will still be present on the final cert.
QuoteHad the same issue on a wildcard cert. Solved it by removing the SAN entry.
The SAN value will still be present on the final cert.
You're right, it has something to do with the SAN. For testing i have removed all SAN and the validadion is working again. But removing the SAN, they are also removed from the certificate of course.
But it's kinda wierd: After removing the SAN equal to the domain it worked again. Now i can add the other SAN (e.g. *.domain.com) again and it still seems to work..
So many thanks for the hint @Modaeus!
Sooooo, is this going to be fixed or just ignored? Just want to sort out my options. To be clear, this works PERFECTLY in pfsense.