OPNsense Forum

English Forums => 25.7, 25.10 Series => Topic started by: excavator fidelity on November 02, 2025, 06:33:41 PM

Title: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 06:33:41 PM
I am trying to use some unexposed Unbound options that require referencing a TLS private key. I want to use my Let's Encrypt certificate which I renew each month via the ACME Client service, but there are two issues with just editing /usr/local/etc/unbound.opnsense.d/custom.conf.


I had run into this same issue with the Prometheus exporter plugin, and I hope that I'm just missing something.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 06:44:39 PM
To avoid an XY problem (https://en.wikipedia.org/wiki/XY_problem), I want to allow clients to connect to unbound via DoT, with a config file that looks like this:

server:
  interface: 192.168.0.1@853
  tls-port: 853
  tls-service-pem: /path/to/public/cert.pem
  tls-service-key: /path/to/private/cert.key
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 08:04:44 PM
This works just fine; that's my custom conf include:

server:
  interface: 2001:db8:1:53::1@853
  interface: 2001:db8:1:53::1@443
  tls-service-key: "/var/etc/acme-client/keys/0123456789abcd.12345678/private.key"
  tls-service-pem: "/var/etc/acme-client/certs/0123456789abcd.12345678/fullchain.pem"

Just look up what "0123456789abcd.12345678" is for your certificate. This path won't change when the certificate renews.

Cheers
Maurice
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 08:34:43 PM
Hmm, thanks for the information but this still isn't working for me.
This may be an Unbound issue and not an OPNsense one, but I can't connect to the server.

$ dig @dns.example.com +tls google.com
;; Connection to 192.168.0.1#853(192.168.0.1) for google.com failed: timed out.
;; no servers could be reached

and on Unbound:

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:0] debug: outnettcp got tcp error -1

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:0] debug: outnettcp cb

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:0] debug: close fd 37

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:2] debug: outnettcp got tcp error -1

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:2] debug: outnettcp cb

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:2] debug: close fd 47

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:0] debug: outnettcp got tcp error -1

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:0] debug: outnettcp cb

2025-11-02T13:33:36-06:00
Debug
unbound
[16668:0] debug: close fd 40
...

# Config
server:
  # I have a static ULA setup in Virtual IPs
  interface: fd00:abcd::1@853
  interface: 192.168.0.1@853
  tls-service-key: "/var/etc/acme-client/keys/.../private.key"
  tls-service-pem: "/var/etc/acme-client/keys/.../fullchain.pem"
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 08:51:27 PM
Hm, this has been working for me for years on two separate OPNsense instances.

Do your firewall settings allow access to 192.168.0.1:853 TCP?

Did you try a basic TLS connection test (on a client in the LAN as well as on OPNsense itself)?

openssl s_client -connect 192.168.0.1:853
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 09:00:14 PM
Quote from: excavator fidelity on November 02, 2025, 08:34:43 PM  tls-service-key: "/var/etc/acme-client/keys/.../private.key"
  tls-service-pem: "/var/etc/acme-client/keys/.../fullchain.pem"

There's a typo in the tls-service-pem path (must be acme-client/certs, not acme-client/keys).
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 09:02:32 PM
$ # This is on the OPNsense box
$ openssl s_client -connect 192.168.0.1:853
0810A5FFB6220000:error:8000003D:system library:BIO_connect:Connection refused:/usr/src/crypto/openssl/crypto/bio/bio_sock2.c:125:calling connect()
0810A5FFB6220000:error:10000067:BIO routines:BIO_connect:connect error:/usr/src/crypto/openssl/crypto/bio/bio_sock2.c:127:
connect:errno=61

That config typo is just from copying into the forum, my config correctly uses the `/var/etc/acme-client/certs/.../fullchain.pem` path.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 09:05:48 PM
Unbound is actually running and works fine when querying it using plain UDP / TCP on port 53?
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 09:09:29 PM
Using sockstat -l I realize that unbound isn't listening on port 853. I assume that means the configuration is not being loaded: is `extra-config.conf` not a valid filename for the `opnsense.unbound.d` directory? It is correctly moved into the chroot.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 09:18:25 PM
Custom includes are put in /usr/local/etc/unbound.opnsense.d/ get copied to /var/unbound/etc/ when Unbound reconfigures. Do the contents of these directories match?

root@router:~ # ls -l /usr/local/etc/unbound.opnsense.d/
total 20
-rw-r--r--  1 root wheel 127 Oct 22 08:53 README
-rw-r--r--  1 root wheel 313 Nov  2 20:41 access_lists.conf
-rw-r--r--  1 root wheel  66 Nov  2 20:41 domainoverrides.conf
-rw-r--r--  1 root wheel 976 Nov  2 20:41 dot.conf
-rw-r--r--  1 root wheel 253 Jan 30  2024 dot_doh_downstream.conf
-rw-r--r--  1 root wheel   0 Nov  2 20:41 safesearch.conf
root@router:~ # ls -l /var/unbound/etc/
total 16
-rw-r-----  1 unbound unbound 313 Nov  2 20:41 access_lists.conf
-rw-r-----  1 unbound unbound  66 Nov  2 20:41 domainoverrides.conf
-rw-r-----  1 unbound unbound 976 Nov  2 20:41 dot.conf
-rw-r-----  1 unbound unbound 253 Nov  2 20:41 dot_doh_downstream.conf
-rw-r-----  1 unbound unbound   0 Nov  2 20:41 safesearch.conf

dot_doh_downstream.conf is my custom include.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 09:25:44 PM
Yes, these directories match exactly (including the contents).

Where are the unbound logs located? I've been using the UI since there isn't a /var/log/unbound directory.
I think it could be an issue where the ACME private key is not readable (since it's owned by root:wheel), but I can't see any errors in the UI log. I believe unbound is started as root though so maybe this isn't a problem.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 09:33:53 PM
root@router:~ # ls -l /var/etc/acme-client/keys/0123456789abcd.12345678/
total 4
-rwxr-x---  1 root wheel 288 Oct 25 00:02 private.key
root@router:~ # ls -l /var/etc/acme-client/certs/0123456789abcd.12345678/
total 12
-rwxr-x---  1 root wheel 1330 Oct 25 00:02 cert.pem
-rwxr-x---  1 root wheel 1567 Oct 25 00:02 chain.pem
-rwxr-x---  1 root wheel 2897 Oct 25 00:02 fullchain.pem

And just in case it matters, my DoT / DoH certs are from Let's Encrypt, no Alt Names, ec-384, OCSP Must Staple disabled.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 10:07:56 PM
Exactly the same certificate profile here.
I checked with `openssl` that the files are correct and have an EC key and x509 cert (which I'm using for the Web UI so it is valid).

I can't get `service unbound restart` to work because the rc.d script assumes I'm in the chroot; I want to try and restart unbound from the shell instead of using the UI to make sure that isn't a source of issues.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 10:39:46 PM
Maybe I'm just too unfamiliar with OPNsense internals, but I was able to get DoT working by directly editing /usr/local/etc/unbound/unbound.conf and then running `sudo service unbound onestart`. But now I have two separate `unbound` instances running!
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 11:14:22 PM
Quote from: excavator fidelity on November 02, 2025, 10:07:56 PMI want to try and restart unbound from the shell instead of using the UI to make sure that isn't a source of issues.

# configctl unbound restart

Quote from: excavator fidelity on November 02, 2025, 10:39:46 PMI was able to get DoT working by directly editing /usr/local/etc/unbound/unbound.conf

This file isn't used by OPNsense, it dynamically creates and uses /var/unbound/unbound.conf. You might want to post the contents of this file. I'm running out of ideas...
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 11:36:53 PM
This is /var/unbound/unbound.conf:

##########################
# Unbound Configuration
##########################

##
# Server configuration
##
server:
chroot: /var/unbound
username: unbound
directory: /var/unbound
pidfile: /var/run/unbound.pid
root-hints: /var/unbound/root.hints
use-syslog: yes
port: 53
include: /var/unbound/advanced.conf
harden-referral-path: no
do-ip4: yes
do-ip6: yes
do-udp: yes
do-tcp: yes
do-daemonize: yes
so-reuseport: yes
module-config: "python iterator"
num-threads: 4
msg-cache-slabs: 8
rrset-cache-slabs: 8
infra-cache-slabs: 8
key-cache-slabs: 8




# Interface IP(s) to bind to
interface: 0.0.0.0
interface: ::
interface-automatic: yes



# Private networks for DNS Rebinding prevention (when enabled)


# Private domains (DNS Rebinding)
include: /var/unbound/private_domains.conf

# Static host entries
include: /var/unbound/host_entries.conf

# DHCP leases (if configured)


# Custom includes
include: /var/unbound/etc/*.conf



python:
python-script: dnsbl_module.py

remote-control:
    control-enable: yes
    control-interface: 127.0.0.1
    control-port: 953
    server-key-file: /var/unbound/unbound_server.key
    server-cert-file: /var/unbound/unbound_server.pem
    control-key-file: /var/unbound/unbound_control.key
    control-cert-file: /var/unbound/unbound_control.pem

And here's /var/unbound/etc/extra_config.conf:

server:
  # Listen on the second loopback interface
  # DNS-over-TLS
  interface: fd53:5353:0000::1@853
  interface: 192.168.0.1@853

  tls-port: 853

  # Let's Encrypt certificate
  tls-service-key: "/var/etc/acme-client/keys/abcd.1234/private.key"
  tls-service-pem: "/var/etc/acme-client/certs/abcd.1234/fullchain.pem"

The acme files exist, the addresses are listed in ifconfig, and I watched my Firewall logs and can confirm my Pass rule works.
I tried renaming the file to start with `00-` and to start with `zz-`.
No matter what, I still have the output of ` sockstat -l | grep unbound` only show listeners on `*:53` (and the local control port `:953`).
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 11:50:25 PM
The only significant difference seems to by that I selected a single interface in "Services: Unbound DNS: General: Network Interfaces" (lo1, which I also use for DoT / DoH).

This is how that looks like in /var/unbound/unbound.conf:

# Interface IP(s) to bind to
interface: 127.0.0.1
interface: ::1
interface: fe80::1%lo0
interface: fe80::1%lo1
interface: 2001:db8:1:53::1

Maybe binding to 0.0.0.0 / :: prevents Unbound from additionally binding to specific addresses for DoT? Worth a try to select only specific interfaces.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: excavator fidelity on November 02, 2025, 11:54:28 PM
THANK YOU! That worked!

I have no clue why unbound works like that, but thanks for helping me work though this issue.
Title: Re: Referencing AcmeClient Certificates from config files
Post by: Maurice on November 02, 2025, 11:58:47 PM
Phew, glad it works! I wouldn't even rule out that the reason why I selected a single interface was this exact issue... This was years ago so I really don't remember.