OPNsense version: 25.7.7_2
os-caddy version: 2.0.4_1
caddy-custom version: 2.10.2.3.0.4.5.4_9
Relevant section from the system log:
2025-11-06T16:53:42-05:00 opnsense.local kernel - - Starting caddy...
2025-11-06T16:53:42-05:00 opnsense.local kernel - - We trust you have received the usual lecture from the local System
2025-11-06T16:53:42-05:00 opnsense.local kernel - - Administrator. It usually boils down to these three things:
2025-11-06T16:53:42-05:00 opnsense.local kernel - -
2025-11-06T16:53:42-05:00 opnsense.local kernel - - #1) Respect the privacy of others.
2025-11-06T16:53:42-05:00 opnsense.local kernel - - #2) Think before you type.
2025-11-06T16:53:42-05:00 opnsense.local kernel - - #3) With great power comes great responsibility.
2025-11-06T16:53:42-05:00 opnsense.local kernel - -
2025-11-06T16:53:42-05:00 opnsense.local kernel - - For security reasons, the password you type will not be visible.
2025-11-06T16:53:42-05:00 opnsense.local kernel - -
2025-11-06T16:54:10-05:00 opnsense.local configctl 11637 - event @ 1762466050.31 msg: Nov 6 16:54:10 opnsense.local config[95297]: config-event: new_config /conf/backup/config-1762466050.2954.xml
2025-11-06T16:54:10-05:00 opnsense.local configctl 11637 - event @ 1762466050.31 exec: system event config_changed response: OK
2025-11-06T16:54:31-05:00 opnsense.local sudo 58583 - www : user NOT in sudoers ; TTY=ttyv0 ; PWD=/ ; USER=root ; COMMAND=/usr/bin/tee /usr/local/etc/ssl/certs/Caddy_Local_Authority_-_2025_ECC_Root_*****.crt
2025-11-06T16:54:31-05:00 opnsense.local kernel - - Password:
2025-11-06T16:54:31-05:00 opnsense.local kernel - - Error: Caddy failed to start
2025-11-06T16:54:31-05:00 opnsense.local kernel - - Check the caddy log: /var/log/caddy/caddy.log
In the console, the boot process just seems to hang where although I can enter characters and press enter, the prompt is unresponsive. If I login via the web interface and disable caddy, then as you can see in the log caddy fails to start and boot continues.
Running caddy as root does not have this problem as it is able to run sudo.
Can anyone please let me know if it is the expected behavior before I try to configure the sudoers file and make it possibly worse?
Apparently this is a known issue (https://github.com/caddyserver/caddy/issues/4248) with Caddy, and it should be documented in the documentation because if someone configures it to run as www, they might be in a bad surprise the next time it reboots and hangs unexpectedly which is what happened to me.
Still would like to know if someone has an answer in case the plugin has a built-in workaround for this issue in which case it might not have properly applied for me for some reason.
I have added the www mode like a year back and lots of people seem to run it and I also tested it a lot also after reboots and never encountered this.
Can you give exact instructions for a reproduction, and open a issue here?
https://github.com/opnsense/plugins
My assumption is that when caddy is started once as root at least, it can install the root certificate and the issue cannot be encountered anymore?
Huh:
> 2025-11-06T16:54:31-05:00 opnsense.local kernel - - Password:
That's probably not working if it's expecting someone to type a password. But where should that happen? The boot runs as root and there is no "sudo" use in the caddy plugin at all. And core also doesn't ever use "sudo" to execute something. In fact, sudo is not part of a FreeBSD base system so I highly doubt the caddy package/rc glue is doing it either.
That only leaves caddy itself or a local modification of some sort.
Cheers,
Franco
I found out how it happens:
- Configure caddy as www user right away without ever running it as root
- Create a domain like "https://abc.internal"
- Reboot
- On reboot caddy will try to install a root certificate for its own CA because it tries to generate a cert for ".internal" and the sudo message appears during boot.
Very specific and weird. :=)
Quote from: Monviech (Cedrik) on November 07, 2025, 08:48:19 AMI found out how it happens:
- Configure caddy as www user right away without ever running it as root
- Create a domain like "https://abc.internal"
- Reboot
- On reboot caddy will try to install a root certificate for its own CA because it tries to generate a cert for ".internal" and the sudo message appears during boot.
Very specific and weird. :=)
Interesting. Yes, I use Caddy both for internal and external domains. I guess it is my internal domain that triggers it. But the thing is Caddy was already configured to run as root initially and I enabled the www mode afterwards. It tries to install its own root certificate every boot even when configured to run as root.
A potential workaround is allowing the execution of two necessary commands via sudoers:
www ALL=(root) NOPASSWD: /usr/bin/tee /usr/local/etc/ssl/certs/Caddy_Local_Authority_-_20??_ECC_Root_*.crt
www ALL=(root) NOPASSWD: /usr/sbin/certctl rehashIdeally, it should only install its root certificate only once. Not sure why it renews it every single time.
https://github.com/smallstep/truststore/blob/d71bcdef66e239112d877b3e531e1011795efdf7/truststore_freebsd.go#L97
Thats where the sudo seems to come from.
For the rest, that must be figured out still.
OK. Apparently Caddy creating its own ephemeral root certificates and attempting to install them is expected behavior (https://caddyserver.com/docs/automatic-https#local-https) but it can be disabled in the configuration.
I disabled it since I don't need it. Here is how:
Add to /usr/local/etc/caddy/caddy.d/disable-install-trust.global:
skip_install_trust
Now OPNsense boots without interruption when caddy set to run as www.
Yeah that directive can indeed disable it.
Let me think about what to do here, it doesnt seem too urgent.
Though Im not sure anybody would need the root cert on the firewall since there is no browser that can be used for it.
So going for that directive might be best for us.
A note in the documentation or in the help text might be useful in case someone uses Caddy for internal domains and decides to enable www mode and reboots where the interruption can be a problem as the firewall partially functions without fully completing the boot process which might mislead the user.
Not needed, we will patch this in the Caddyfile next week.
Installing the root cert should never be needed since there is no browser or anything that can use it on a firewall.
https://github.com/opnsense/plugins/issues/5011