ACME client issues w/Cloudflare

[DenverTech]

I've seen and read many posts about issues with Cloudflare, but have been using it without issue for about 1-2 years, using the generated API keys from CF. I use a wildcard domain and all renewals worked from 2022 until about 70 days ago. Then, mysteriously, they stopped working with the errors below. Hoping someone has some ideas on this as I've been beating my head against it for days.

Issue:

• Starting about 70 days ago, the renewals began failing with "invalid domain" and "Error add txt for domain"
• In the past, others have fixed this with updates (I'm current on both OPNsense and plugins) or new API keys (tried that)
• Rebuilt all stages of the cert and issue persists
• Tried with a single subdomain and issue persists

Tested:

• Recreated the verification challenge, as that's where it's failing. Same errors.
• Verified/recreated the API key permissions in case something changed on CF's end. Same errors.
• Switched to a single subdomain, rather than wildcard. Same errors.
• Recreated all stages of the request. Same errors.
• Created a new API key with correct permissions. Same errors.
• Contacted CloudFlare. They blame OPNsense, because of course they do.
• NOTE: The API key does have zone read and dns edit permissions

Code Select Expand

See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
Please add '--debug' or '--log' to check more details.
Error add txt for domain:_acme-challenge.somedomain.com
invalid domain
Adding txt value: <somestring> for domain: _acme-challenge.somedomain.com
Getting webroot for domain='*.somedomain.com'
Getting domain auth token for each domain
Single domain='*.somedomain.com'
Using CA: https://acme-v02.api.letsencrypt.org/directory

[Monviech (Cedrik)]

As sanity check you could try getting the wildcard cert from cloudflare from the plugin in my signature. It has the cloudflare DNS Provider and DNS-01 challenge build in. It uses libdns and this provider https://github.com/caddy-dns/cloudflare

[DenverTech]

I really don't want to learn Caddy to fix an issue that just cropped up with the built-in system. I'll consider that a last resort.

Side-note...tested again using the global API key. Also says the domain is invalid.

[DenverTech]

Lacking other options, I did try the Caddy plugin. No luck...but different results.

Example, it's setup with some.sitename.com points to handler 192.168.0.1, port 1111. I go to some.sitename.com:443 and it gives me a secure blank page. It does not forward to 192.168.0.1:1111 at all.

Progress, maybe? Still would love to know why the built-in plugin isn't working, but no one seems to want to talk about it, judging by the other threads about this. :)

[Monviech (Cedrik)]

Well I guess that means it is possible for you to get Let's Encrypt Certificates with TXT Records of Cloudflare. Right? So that means your API Token and the API of Cloudflare works as expected, and the issue has to be somewhere with the ACME Plugin implementation of it?

[DenverTech]

Does seem to be the case! I definitely didn't mean to break the acme plugin. :D

[Monviech (Cedrik)]

If you have logs of the ACME plugin, you could open an issue on github, maybe theres a fix for it upstream that can be implemented? https://github.com/opnsense/plugins

Sadly I dont know much about how the ACME Plugin works.

[rdunkle84]

I noticed that when creating the cloudflare api token, Acme required:
Zone Resources set: Include | All zones.   This appears to be the problem.
To sum it up:
Zone | DNS | Edit
Zone Resources | Include | All Zones
Client IP (not using this field)
TTL | set a valid date range
This appears to work OK.

[DenverTech]

I noticed that when creating the cloudflare api token, Acme required:
Zone Resources set: Include | All zones.   This appears to be the problem.
To sum it up:
Zone | DNS | Edit
Zone Resources | Include | All Zones
Client IP (not using this field)
TTL | set a valid date range
This appears to work OK.

Tried this. Still says the domain is invalid. I've got all zones allowed and a TTL, as well as the edit permissions.

[opnsenseuser]

I´m using cloudflare too.
After the latest update OPNsense 24.1.4 i get a validation failed error.

[tuananh]

did you find a way to solve problem?

Im still having this issue on latest release (24.1.7)

[julsssark]

I am using 24.1.6, and the Acme plugin with CloudFlare DNS-01 challenge. My certificates are updating as expected and my last certificate updated on May 12. I am using Let's Encrypt as my Acme CA, a restricted API token (zone read, DNS edit) and named certs.

[liceo]

Same problem here, one of my website's cert has expired now!! No clue how to fix and customer already complaining. Running

Code Select Expand

AcmeClient: domain validation failed (dns01)

acme.ch seems to have problems adding the txt, but i can't see why..

Code Select Expand

[Wed May 29 12:54:39 CEST 2024] Add txt record error.

This is geeting urgent!

[liceo]

Some more logs...

Code Select Expand


2024-05-29T14:56:40 opnsense AcmeClient: running acme.sh command: /usr/local/sbin/acme.sh --issue --syslog 8 --debug 2 --server 'letsencrypt' --dns 'dns_cf' --dnssleep '300' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/62b86c7fd6ddb9.24403730' --certpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/cert.pem' --keypath '/var/etc/acme-client/keys/62b86c7fd6ddb9.24403730/private.key' --capath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/chain.pem' --fullchainpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/fullchain.pem' --domain 'mydomain.com' --domain 'mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/5f806aef5d0241.03202364_prod/account.conf'
2024-05-29T14:56:40 opnsense AcmeClient: using challenge type: Cloudflare DNS Validation
2024-05-29T14:56:40 opnsense AcmeClient: account is registered: avbs-acme
2024-05-29T14:56:40 opnsense AcmeClient: using CA: letsencrypt
2024-05-29T14:56:40 opnsense AcmeClient: issue certificate: mydomain.com
2024-05-29T14:56:40 opnsense AcmeClient: certificate must be issued/renewed: mydomain.com
2024-05-29T12:54:44 opnsense AcmeClient: validation for certificate failed: mydomain.com
2024-05-29T12:54:44 opnsense AcmeClient: domain validation failed (dns01)
2024-05-29T12:54:44 opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command returned exit code '1': '/usr/local/sbin/acme.sh --issue --syslog 7 --debug --server 'letsencrypt_test' --dns 'dns_cf' --dnssleep '300' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/62b86c7fd6ddb9.24403730' --certpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/cert.pem' --keypath '/var/etc/acme-client/keys/62b86c7fd6ddb9.24403730/private.key' --capath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/chain.pem' --fullchainpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/fullchain.pem' --domain 'mydomain.com' --domain 'mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/5f806aef5d0241.03202364_stg/account.conf''
2024-05-29T12:54:29 opnsense AcmeClient: running acme.sh command: /usr/local/sbin/acme.sh --issue --syslog 7 --debug --server 'letsencrypt_test' --dns 'dns_cf' --dnssleep '300' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/62b86c7fd6ddb9.24403730' --certpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/cert.pem' --keypath '/var/etc/acme-client/keys/62b86c7fd6ddb9.24403730/private.key' --capath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/chain.pem' --fullchainpath '/var/etc/acme-client/certs/62b86c7fd6ddb9.24403730/fullchain.pem' --domain 'mydomain.com' --domain 'mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/5f806aef5d0241.03202364_stg/account.conf'
2024-05-29T12:54:29 opnsense AcmeClient: using challenge type: Cloudflare DNS Validation
2024-05-29T12:54:29 opnsense AcmeClient: account is registered: avbs-acme
2024-05-29T12:54:29 opnsense AcmeClient: using CA: letsencrypt_test
2024-05-29T12:54:29 opnsense AcmeClient: issue certificate: mydomain.com
2024-05-29T12:54:29 opnsense AcmeClient: certificate must be issued/renewed: mydomain.com


[Monviech (Cedrik)]

If its a customer who is complaining, why not just buy a certificate? Getting a wildcard certificate for the domain/s fixes the problem instantly and it doesn't cost much for a business.