Tutorial: Caddy (Reverse Proxy) + Let's Encrypt Certificates + Dynamic DNS

[yarcod]

Another question for that matter, would it possible to add an option to include files under "caddy.d" inside a domain matcher? In my case it would create a Caddyfile like this:

Code: [Select]

*.edholm.cc {
  # Options
  import /usr/local/etc/caddy/caddy.d/*.conf
  # Or perhaps lookup files with the configured domain as a prefix, like *.edholm.cc.conf
}
Mainly this is to make external automations able to add individual handlers as separate files for an existing domain. Benefit is that the plugin keeps track of the TLS cert and some other FW configured things, and, in my case, Ansible could still add configurations without requiring opening the webui manually. This should not break any existing plugin functionality, I think, except if you add files which handles the same subdomains as the plugin is set up to handle already.

[Monviech (Cedrik)]

Your *.edholm.cc certificate probably doesn't have a SAN for exactly edholm.cc.

Its only valid for subdomains under edholm.cc but not for the domain exactly.

Im not sure why it doesnt work for your setup though, it looks correct in the caddyfile if the cert is really for *.example.com. All manual wildcards I have used before, and also ones generated by Caddy itself always worked. If you have s supported DNS Provider just let caddy handle the wildcard domain generation.

------

If you want to change options remotely, use the REST API the plugin provides. Most modern OPNsense plugins and core features have this API.

e.g., in a browser call /api/caddy/reverse_proxy/get

https://docs.opnsense.org/development/api.html

Main advantage, everything you do will be validated and serialized into the /conf/config.xml so OPNsense backups contain it.

[jrob801]

I'm working on getting Caddy set up on my OPNSense box, and running into some issues that are probably stupid user error from someone who's brand new to the featureset of OPNSense. Hoping someone can help.

My situation: I got Caddy set up with cloudflare pointing to my domain. Was able to obtain a wildcart cert from cloudflare. Set up my subdomains for Plex, Sonarr, HomeAssistant, and router GUI.

I'm getting 502 errors on all of them from the WAN, but can access Plex from the local network (the rest give 502). I can't play anything on Plex, but can see and interact with my dashboard.

As far as my setup goes, here's what I have:

Cloudflare DNS records:

Code: [Select]

ha.example.us. 1 IN A <my WAN IP>
plex.example.us. 1 IN A <my WAN IP>
*.example.us. 1 IN A <my WAN IP>
example.us. 1 IN A <my WAN IP>
router.example.us. 1 IN A <my WAN IP>
sonarr.example.us. 1 IN A <my WAN IP>
I set up the firewall rules as directed in the tutorial, with HTTP and HTTPS rules on both LAN and WAN.

Domain:

Code: [Select]

Protocol: https://
domain: *.example.us
Cert: ACME
DNS-01 Challenge: checked
Dynamic DNS: unchecked
Subdomains:

Code: [Select]

Domain: *.example.us
Subdomain: plex.example.us
Dynamic DNS: checked
Other services are set up the same.

HTTP Handlers:

Code: [Select]

Domain: *.example.us
Subdomain: plex.example.us
directive: reverse_proxy
Protocol: Https://
Upstream Domain: plex server's local IP
Upstream Port: 32400
TLS Insecure Skip Verify: checked
Other services are set up the same, but with correct subdomain, local IP, and port.


Here's a my caddyfile:

Code: [Select]

# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


# caddy_user=root

# Global Options
{
log {
output net unixgram//var/run/caddy/log.sock {
}
format json {
time_format rfc3339
}
}

servers {
protocols h1 h2 h3
}

dynamic_dns {
provider cloudflare <my token>
domains {
example.us plex
example.us router
example.us sonarr
example.us ha
}
}

email myemail@gmail.com
grace_period 10s
import /usr/local/etc/caddy/caddy.d/*.global
}

# Reverse Proxy Configuration


# Reverse Proxy Domain: "14b010dd-6f6d-4ba8-94df-898d341059b8"
*.example.us {
tls {
issuer acme {
dns cloudflare <my token>
}
}

@85082da4-6d7a-410d-9955-d9c114e40692 {
host plex.example.us
}
handle @85082da4-6d7a-410d-9955-d9c114e40692 {
handle {
reverse_proxy 192.168.1.137:32400 {
transport http {
tls
tls_insecure_skip_verify
}
}
}
}
@fc7226be-505c-4b24-b95b-e01cb5e11a32 {
host router.example.us
}
handle @fc7226be-505c-4b24-b95b-e01cb5e11a32 {
handle {
reverse_proxy 192.168.1.1:8443 {
}
}
}
@e857ccbe-2f6a-4d4d-86f5-f8a5ab908e70 {
host sonarr.example.us
}
handle @e857ccbe-2f6a-4d4d-86f5-f8a5ab908e70 {
handle {
reverse_proxy 192.168.1.137:8989 {
transport http {
tls
tls_insecure_skip_verify
}
}
}
}
@fa0581ec-66ca-4526-ae6b-6a059d16f73c {
host ha.example.us
}
handle @fa0581ec-66ca-4526-ae6b-6a059d16f73c {
handle {
reverse_proxy 192.168.1.138:8123 {
}
}
}
}

import /usr/local/etc/caddy/caddy.d/*.conf


Any help here? Do I need to do any setup on my Plex/Sonarr/HA box? Forward ports 443/80? Is unbound DNS tripping me up somehow? Or did I just miss something else?

[Patrick M. Hausen]

Did you move the OPNsense UI to a different port and disable the HTTP --> HTTPS redirect?

[jrob801]

Yes I did. Sorry for forgetting to mention that

[Monviech (Cedrik)]

I think you have to configure "trusted proxies" to a list of cloudflare IPs in General Settings. (if you use it as CDN and not only as DNS provider)

There can also be cloudflare specific settings to be done at cloudflare itself I do not know about. I dont use it sorry. I think Cloudflare can itself be tje reverse proxy entry point for domains configured on it.

Check out what

curl -v example.com

returns from the outside.

[Gautier]

Hi,
I install Caddy and configure follow the tutorial but I have error:
"error","ts":"2024-10-11T07:26:56Z","logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"toto.pequod.sokil.fr","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 89.219.181.98: Timeout during connect (likely firewall problem)"}

I really don't know where to start
I also on freeBSD and debian install caddy to test with the same error.

I have another site with OPNsense and caddy on debian behind without error, I miss something but what ?

Hi,

Just to say I just install opnsense with caddy on another ISP (another country) and it's work like a charm. I ask mu isp to help me but they send a pastry chef so ...

Anyway just to say thank you for your help.

Regards

[Monviech (Cedrik)]

Hey thanks for checking back in. Yeah ISPs are a big factor, some use CGNAT, some block port 80/443 etc..

[jrob801]

I think you have to configure "trusted proxies" to a list of cloudflare IPs in General Settings. (if you use it as CDN and not only as DNS provider)

There can also be cloudflare specific settings to be done at cloudflare itself I do not know about. I dont use it sorry. I think Cloudflare can itself be tje reverse proxy entry point for domains configured on it.

Check out what

curl -v example.com

returns from the outside.

I'm only using cloudflare for DNS to access my home network. My server is inside the network. As far as cloudflare specific settings, to my knowledge there aren't any. I watched several youtube tutorials on setting up cloudflare as DDNS with OPNSense, but unfortunately, I couldn't find anything tied to your plugin (there's really next to no info I've found outside of this thread and a handful of posts you've replied to on Reddit)


Here's the result of the curl entry...

Code: [Select]

* Host sonarr.mydomain.us:80 was resolved.
* IPv6: (none)
* IPv4: <mywanip>
*   Trying <mywanip>:80...
* Connected to sonarr.mydomain.us (<mywanip>) port 80
> GET / HTTP/1.1
> Host: sonarr.mydomain.us
> User-Agent: curl/8.9.1
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://sonarr.mydomain.us/
< Server: Caddy
< Date: Mon, 04 Nov 2024 17:52:26 GMT
< Content-Length: 0
<
* we are done reading and this is set to close, stop send
* shutting down connection #0


[Monviech (Cedrik)]

Well that looks good. Port 80 is caddy answering. Do the same curl for 443 too.

curl -v example.com:443

Yeah the plugin is still less than a year old so I dont expect any other user tutorials yet. Might take some more time. Thats why I wrote these big docs maybe somebody can PR a cloudflare tutorial sometime.

[jrob801]

Code: [Select]

* Host sonarr.mydomain.us:80 was resolved.
* IPv6: (none)
* IPv4: <wanip>
*   Trying <wanip>:80...
* Connected to sonarr.mydomain.us (<wanip>) port 80
> GET / HTTP/1.1
> Host: sonarr.mydomain.us
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://sonarr.mydomain.us/
< Server: Caddy
< Date: Mon, 04 Nov 2024 19:32:57 GMT
< Content-Length: 0
<
* shutting down connection #0
Looks the same as what I got on port 80, other than the "HTTP/1.1 308 Permanent Redirect" line. So if cloudflare is getting through to Caddy, any suggestions as far as what's blocking the request from getting through the firewall?

As for setting up cloudflare, assuming my current issues are on the firewall side, there's nothing special about the setup. You just have to create an A record for your root domain, or using * to get a wildcard cert. I didn't even create the DNS entries for my subdomains, cloudflare did it dynamically.

[Monviech (Cedrik)]

I think you have to curl -v https://example.com

So far we have just seen port 80 open but never port 443. If connections time out externally its 443 that could be the issue.

If 443 is indeed open and caddy answers and there are no timeouts then its weird.

[jrob801]

Apparently my brain isn't turned on yet today. I didn't even notice that both tests were for port 80. Here's 443

Code: [Select]

curl -v sonarr.example.us:443
* Host sonarr.example.us:443 was resolved.
* IPv6: (none)
* IPv4: <wanip>
*   Trying <wanip>:443...
* Connected to sonarr.example.us (<wanip>) port 443
> GET / HTTP/1.1
> Host: sonarr.example.us:443
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
* HTTP 1.0, assume close after body
< HTTP/1.0 400 Bad Request
<
Client sent an HTTP request to an HTTPS server.
* shutting down connection #0


[Monviech (Cedrik)]

Yeah well that works too, I would have tested with https://example.com instead of only using example.com:443 but this shows the port is open and theres at least some answer. It is not clear what answers though.

Just try to fiddle a bit around and see what responds. Im unable to do this step by step here in this thread. Also check out the tutorial section here that explains what should happen if things work.

Your Caddyfile looks totally fine so it has to be some networking issue.

https://docs.opnsense.org/manual/how-tos/caddy.html#help-nothing-works

[jrob801]

I ran the curl command a few more ways and got some results that seem mildly interesting. Figured I'd repost them in case they're telling to you...

Code: [Select]

C:\Users\me>curl -v mydomain.us
* Host mydomain.us:80 was resolved.
* IPv6: (none)
* IPv4: <wanIP>
*   Trying <wanIP>:80...
* Connected to mydomain.us (<wanIP>) port 80
> GET / HTTP/1.1
> Host: mydomain.us
> User-Agent: curl/8.9.1
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://mydomain.us/
< Server: Caddy
< Date: Mon, 04 Nov 2024 21:16:01 GMT
< Content-Length: 0
<
* we are done reading and this is set to close, stop send
* shutting down connection #0


Code: [Select]

C:\Users\me>curl -v https://mydomain.us
* Host mydomain.us:443 was resolved.
* IPv6: (none)
* IPv4: <wanIP>
*   Trying <wanIP>:443...
* Connected to mydomain.us (<wanIP>) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed). More detail may be available in the Windows System event log.
* closing connection #0
curl: (35) schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed). More detail may be available in the Windows System event log.


Code: [Select]

C:\Users\me>curl -v https://www.mydomain.us
* Host www.mydomain.us:443 was resolved.
* IPv6: (none)
* IPv4: <wanIP>
*   Trying <wanIP>:443...
* Connected to www.mydomain.us (<wanIP>) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.x
> GET / HTTP/1.1
> Host: www.mydomain.us
> User-Agent: curl/8.9.1
> Accept: */*
>
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 200 OK
< Alt-Svc: h3=":443"; ma=2592000
< Server: Caddy
< Date: Mon, 04 Nov 2024 21:19:35 GMT
< Content-Length: 0
<
* Connection #0 to host www.mydomain.us left intact

So it looks like a request to the bare domain responds over http/port 80, a bare request trying to force https results in a TLS handshake error, but a https request to www works.  That's particularly interesting to me since I didn't set up a www entry in my DNS record.

Is there anything noteworthy you're seeing from those results?

I'm running through the troubleshooting steps and reviewing firewall rules etc to see if I can find something on the firewall end that's stopping the request from passing through to the client...