ACME NGINX HTTP 400 Again.

[itngo]

We had this issue in the past, but now it is ongoing for 2 days for a newly created Certificate. Existing Certs with HTTP-Challenge are working for renewal. We just copied and existing Cert and also a WEB in NGINX from a working one.

We can Download the challenge-file with any browser from any side. (Using DEBUG-Mode 3 so file does not get deleted)
However Let's Encrypt says

{
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:ietf:params:acme:error:connection",
    "detail": "removed but is correct: Fetching http://removed but is correct/.well-known/acme-challenge/znc28dKFOGTaUY1o8GW1gbtQ_o40aecyyAmhXT-ur8g: Timeout during connect (likely firewall problem)",
    "status": 400
  },
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/removed but is correct/seTRuQ",
  "token": "removed but is correct",
  "validationRecord": [
    {
      "url": "http://removed but is correct/.well-known/acme-challenge/znc28dKFOGTaUY1o8GW1gbtQ_o40aecyyAmhXT-ur8g",
      "hostname": "removed but is correct",
      "port": "80",
      "addressesResolved": [
        "remove but is correct"
      ],
      "addressUsed": "removed but is correct"
    }
  ],
  "validated": "2022-10-18T06:00:33Z"
}

Any ideas what to do next?

[itngo]

When I compare "rights" of an working and a non working acme-challenge file after starting renewal I can see a difference. Is that normal? See attachment....

[itngo]

Created an ZeroSSL-Account had repeated the Issue/Renewal. It works. So something with the Let's Encrypt Challenge for HTTP-01 is not working here.

What can we do? Any suggestions?

[Fright]

what is set in "HTTP Service" dropdown in HTTP-01 challenge settings?
any Port Forward rules involving tcp80?

[itngo]

Hi,
see attachment...

no port forwarding. About 200 Webs are running with this config and about 50 Certs already issued with let's...

[Fright]

Hi)
I would look at the HAProxy logs in this case

[itngo]

Ok, but why? We use nginx and have the Lets-Encrypt-Integration enabled and can see that the http request is reaching the NGINX-Server when we try to validate. So indeed the Lets-Servers can connect but do respond with 400 error.

[Fright]

sorry, maybe I just didn't fully understand your configuration: why is HAProxy specified as a HTTP Service and not the default "OPNSense.." value is used?
How is this scheme supposed to work? (Is the "Enable Let's Encrypt Plugin Support" for the specified server enabled in nginx? Or are the files placed in some other way?)
Let's Encrypt Error 400 is not very self-explanatory: for example, it can even be a dna issues like
https://community.letsencrypt.org/t/error-lets-encrypt-validation-status-400/99289

[itngo]

We use both on the opnsense. NGINX and HAProxy because NGINX can not do NTLM-Forward while HAProxy can.
So we use both to configure the most secure. NGINX with NAXSI for some Webs and HAProxy for other Webs.

Both on different IPs and bindings. It is working, except one Let's Cert which we created recently. This one can not issue and gives the so called 400. However, it is exactly configured like about 45 other Certs.

I know, 400 can mean everything or nothing. Had this in the past and it was just a matter of waiting some hours. But this is ongoing for days now...

[muchacha_grande]

May be this fix the problem:

https://forum.opnsense.org/index.php?topic=30656.msg148080#msg148080

Cheers

[Fright]

@itngo
hi. sorry for the delay.
OK, I think I understand your settings. if it is assumed that the configuration itself (without using the default acme settings) is correct and working, then the only thing that comes to my mind is "authorization cache" vs. "Multi-Perspective Validation": validation may not be at all when renewing the certificate (if validation cache ttl is not exceeded) and it is possible that the firewall blocked one of the validation requests when a new one is issued (https://letsencrypt.org/2020/02/19/multi-perspective-validation.html)

[itngo]

We will test that again in about 30 days, to make sure there is nothing in the cache anymore....

Thank you....