Tutorial: Caddy (Reverse Proxy) + Let's Encrypt Certificates + Dynamic DNS

[Monviech (Cedrik)]

Actually the Layer 4 feature in Caddy had a big overhaul, so you can use it to stream UDP traffic too, from any port to any port. And you can even use layer 7 matchers additionally inside the layer 4 matchers to selectively match and route traffic.

Check it out

[Patrick M. Hausen]

Actually the Layer 4 feature in Caddy had a big overhaul, so you can use it to stream UDP traffic too, from any port to any port.

TIL - thanks

[Monviech (Cedrik)]

@Patrick

E.g. you can match and route wireguard which is UDP

https://github.com/opnsense/plugins/issues/4201#issuecomment-2405077886

[Baender]

Thank you for the explanation.

I have noticed, that since I enabled IPv6 and using Track Interface, most of my subdomains, that are restricted to local IPs, won't work. This is because I only set IPv4 addresses to the access list. How can I allow "local" IPv6 addresses?

[Monviech (Cedrik)]

You just have to add your GUA to that access list.

e.g.

2001:db8:1234:5600::/56

If its not static you are a little out of luck here since the access list can not be updated with a dynamic prefix. In that casw it would be easier to only have A Records and no AAAA records for your domains.

If you use the dynamic dns feature set it to ipv4 only.

[Baender]

Oh, no. Please don't tell me that.. Yes, it is dynamic.. Could I override the Caddyfile then? I know, that for Caddy there might be such Plugin, but if I would write a script, that checks the current IP? Would that help?

Just a side note from my side: with one of the recent caddy releases on OPNsense, the description for domains and subdomains got a bit out of control. I personally find it problematic, to show the domain and the description, that results in a lot of text, in the selection GUI and on the overview. Do you know what I mean?

[Monviech (Cedrik)]

The issue here is how to tell caddy that the IP has to change in the access list, and then also reload the service so it is indeed loaded. This can be rather complex, since there is no general default access list into which this could be implemented genericly. Everybody manages their own lists that are saved in their own configuration.

There is no official module for this: https://github.com/caddyserver/caddy/issues/5813

And regarding the description, I had to add it to the subdomain dropdown since otherwise there would be no way to keep different subdomains apart, that have the same subdomain name, but are added under multiple wildcard domains with different ports. So we have to life with that now.

[Monviech (Cedrik)]

If you want to write a script it has to be a php script that targets the API that the framework of the caddy plugin provides. You could update your specific access list witj your correct IPv6 network when it changes and then use the service API to trigger a reconfigure act.

https://github.com/opnsense/plugins/tree/master/www/caddy/src/opnsense/mvc/app/controllers/OPNsense/Caddy/Api

[Monviech (Cedrik)]

All descriptions in domains/subdomains etc... are optional btw so delete the ones you dont need.

[Baender]

Okay a really complex topic then. Would it be (in theory) possible, to let Caddy read the aliases of the firewall? Would it be a polling, or as it might be in the memory, on the fly change, if I would add the IPv6 address as an alias?

I guess, still out of scope. However, I am just curious.

Edit:
Would it be possible, to set dynamic DNS setting on the domain/subdomain? I mean if it should pass a IPV6 and/or IPv4 address? Maybe the module for dynDNS is not structured for this..

[dinguz]

I have caddy configured as a reverse proxy for the OPNsense GUI and as a SSH multiplexer, more or less according to the docs found here: https://github.com/opnsense/docs/blob/11e66816989bb12633e01e144ebf42b11508755a/source/manual/how-tos/caddy.rst

Unfortunately, I am having an issue where it prevents system reboots, both from the GUI and from the CLI menu (option 6). The rebooting hangs on shutting down caddy, which apparently never happens. The only thing that currently works is issue a 'shutdown -r now' from the CLI. The log files aren't giving me much direction how to go about investigating this. Any tips or advice?

[Monviech (Cedrik)]

@dinguz

I have to ask upstream for this. I'll come back to this when I know more. What I can say is I do sometimes fight with the shutdown behavior of caddy. When it has open tcp/udp connections it can wait indefinitely until they are closed in certain configurations.

Thats why the service script now has sigkill implemented. But that will not happen on "service caddy stop" (which is used during reboot), only on "configctl caddy stop".
https://github.com/opnsense/plugins/pull/4261


@Baender

The dynamic dns module only has global settings. So set Ipv4 only globally if you need access lists with rfc1918 addresses. Otherwise look into forward auth or basic auth to secure access if you rely on dynamic ipv6. The way access lists are handled will not be changed in the current plugin implementation.

[Monviech (Cedrik)]

@dinguz

https://github.com/mholt/caddy-l4/issues/258

For now either close the ssh connection after initiating a restart, or use "configctl caddy stop" since it has a mechanism that will kill caddy after 20 seconds.

[dinguz]

For now, either close the ssh connection after initiating a restart, or use "configctl caddy stop" since it has a mechanism that will kill caddy after 20 seconds.

Many thanks!

[meyergru]

Today I took the opportunity to try out Caddy reverse proxy instead of HAproxy, mostly because of a very specific problem with HAproxy...

I must say I reverted after trying it thouroughly. My 2cents on this are as follows:

- Caddy is suited to home setups and inexperienced users. HAproxy is much more complex.
- For example, the certificate setup is much easier, because you just have to specify the domain and it just works (tm).
- However, if you have more than just one domain, Caddy setup gets a little tedious:
* you have to create one domain/certificate plus a http backend for any domain, which includes creating different ones for www.domain.de and domain.de. You cannot combine certificates for multiple domains unless they are subdomains.
* You do not have much control over what type of certificate(s) are created - you cannot specifiy strength or ECC vs. RSA (much less both) and I have not found a means to control if ZeroSSL vs. LetsEncrypt is used.
* The ciphers being employed cannot be controlled easily - or, for TLS 1.3, at all. That results in an ssllabs.com score which is suboptimal, becaus 128bit ciphers are allowed. This cannot be changed because of Go limitations.
* You cannot use more than one type of DNS-01 verification if you use wildcard domains.
* The Auto HTTPS feature looks nice first, but indeed it uses a 308 instead of a 301 code, which breaks some monitoring and can only be modified via custom include files.

So, if you just want to reverse-proxy some services in your home network, go with Caddy. For an OpnSense guarding your internet site with several services/domains, stay with HAproxy.