Acme HTTP-01 challenge failure

[yodaphone]

I have a VM running ubuntu and I'm trying to obtain a cert via http-01 challenge
i have forwarded the port 80 & 443 to the IP address of the VM and also setup Unbound & DNS (cloudflare) to point to my WAN address.

When i try to obtain a certificate using certbot on the VM, it errors out with this message. Any idea why this is failing.

The application I am using has some issues with using reverse proxies and hence I had to obtain a cert directly. However, I also have caddy configured on my firewall & if i get a cert using DNS-01 challenge, that works for fine for the same domain.

Any idea why I'm getting the " Timeout during connect (likely firewall problem)" error

Code Select Expand

sudo certbot --nginx --preferred-challenges http --agree-tos --no-eff-email -v --staging -d rust.mydomain.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Requesting a certificate for rust.mydomain.com
Performing the following challenges:
http-01 challenge for rust.mydomain.com
Waiting for verification...
Challenge failed for domain rust.mydomain.com
http-01 challenge for rust.mydomain.com

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: rust.mydomain.com
  Type:   connection
  Detail: x.x.x.x: Fetching http://rust.mydomain.com/.well-known/acme-challenge/7RkJ4hMZ8-muJIFtuJ2nz4XnLVdK0TF7t8tkebRG-Xk: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.


Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

2025-04-14 11:45:16,282:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2025-04-14 11:45:16,282:DEBUG:certbot._internal.error_handler:Calling registered functions
2025-04-14 11:45:16,282:INFO:certbot._internal.auth_handler:Cleaning up challenges
2025-04-14 11:45:17,437:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==2.9.0', 'console_scripts', 'certbot')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 1450, in run
    new_lineage = _get_and_save_cert(le_client, config, domains,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line 143, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2025-04-14 11:45:17,443:ERROR:certbot._internal.log:Some challenges have failed.

The LiveView in Firewall logs appears to pass the traffic

[meyergru]

How did you forward ports 80 & 443? Via Caddy as reverse proxy (because you say you "also have caddy configured on my firewall") or via port forwarding?

If it is the former, then Caddy would most likely catch the "/.well-known/acme-challenge" URI before your VM backend can handle it. While you can used DNS-01 verification, this probably does not disable HTTP-01 verification.

Thus, you should either do all certificates on the frontend or none.

[Monviech (Cedrik)]

Caddy can reverse proxy the HTTP-01 challenge for you (without port forwards needed), described here in the docs:

https://docs.opnsense.org/manual/how-tos/caddy.html#redirect-acme-http-01-challenge