ACME Cert Renewals

[gazd25]

Hi Guys,

On my up to date OPNsense 23.1.9-amd64 firewall, I've noticed that my ACME certificate renewals are both now showing as failed validation in the logs as below:

2023-06-12T14:32:53   acme.sh   [Mon Jun 12 14:32:53 BST 2023] Error add txt for domain:_acme-challenge.contoso.com
2023-06-12T14:32:53   acme.sh   [Mon Jun 12 14:32:53 BST 2023] invalid domain

I cant see much history in the logs but it seems to have showed the same error for the last few renewal attempts which happen at midnight automatically.

The ACME renewal process uses the Cloudflare DNS validation method and no config changes have been made at all. Until recently this has always worked very well for me without issues.

I did run an update this morning and noticed a new ACME script was brought down, so wondered if there has been any changes which might have impacted?

I also tried to force renew and noted that the extra text record never appears in Cloudflare DNS as expected, so it does appear to be some change, but it's difficult to say for sure.

I've got a snapshot and rollback capability so am going to try a few different things in testing, but thought it was first worth raising to see if it's just me.

Thanks for any help.

Gareth

[gazd25]

Well,

As is always the best way, I solved my own problem.

There was some changes a little while back, related to my IPv6 configuration, as a result of a change to the PPPoE initiation in 23.1.7 version of OPNSense. For more detail on that, see here:

https://forum.opnsense.org/index.php?topic=33864.45

In any case, after trying pretty much everything else I could think of, I began investigating the Cloudflare API as a possible culprit for the failure to renew on the ACME client.

Turned out, I had locked down the API calls to a specific token allowing only my old static IPv4 and IPv6 addresses to make the request, the IPv6 of course has now changed because of moving to PPPoEv6 on my WAN interface.

It seems my firewall was using IPv6 to contact the Cloudfare API and then not falling back to IPv4 if the request failed due to the API controls disallowing the request, also leading to a fairly nonsensical error being logged on OPNSense which bore no resemblance to what was actually going on.

In any case, I did a little testing to ensure I knew which of my firewalls IPv6 addresses the Cloudflare API was receiving the request from, altered the API token settings on Cloudflare to allow this IPv6 to initiate the requests to the API and hey presto, my certificates are now both renewing correctly again.

Hope my little journey helps somebody else in the future

Thanks

Gareth