Caddy Layer4 Route TLS (SNI) stopped working

Started by ept, Today at 05:17:11 PM

Previous topic - Next topic
Hello!

I used the Caddy plugin with its layer 4 functionality solely to forward incoming HTTP and HTTPS requests depending on the server name, without terminating TLS. This worked great from early 2025 until May of this year, when - after apparently some browser updates - it stopped working. Since then I've just used the "catch all" TLS forwarding instead of TLS (SNI).

The error seems very similar to the one described in the post https://forum.opnsense.org/index.php?topic=42955.0 , which was resolved then. (And it worked well for me until May.)

More precisely, I see the following behavior:
- When I use curl for the request it works. (That is, the request gets forwarded correctly and I get an answer without any error.)
- When `security.tls.enable_kyber` is deactivated in Firefox, it works also.
- With the default settings of Firefox and Chromium my sites don't load but an SSL error is displayed, and I get log entries in Caddy like this:
    "debug","ts":"2026-09-06T13:21:22Z","logger":"caddy.listeners.layer4","msg":"matching","remote":"123.123.123.123:34567","error":"consumed all prefetched bytes","matcher":"layer4.matchers.tls","matched":false}

I currently use OPNsense 26.7.3_11 (but the described behavior started before the update to 26) and the caddy plugin 2.2.1 (Caddy 2.11.4, caddy-l4 0.1.2).

Thanks
Eric

This appears to be the already identified caddy-l4 routing bug described in PR #455, rather than merely another insufficient prefetch-buffer size.

If a TLS matcher needs more data because the ClientHello arrived in multiple TCP segments, a subsequent HTTP route can reach a definitive non-match and inadvertently suppress the pending TLS match. Post-quantum ClientHellos make this much more reproducible because they commonly span multiple TCP segments.

This fits your observations exactly: curl and Firefox without Kyber send a smaller ClientHello, whereas current Firefox and Chromium trigger the bug.

As a workaround, try placing all Layer 4 HTTP routes before all TLS/SNI routes. PR #455 specifically identifies that route ordering as a workaround and states that caddy-l4 0.1.2 is affected. IDK if that is possible via the GUI, though... otherwise, Deciso would have to either include the PR or change configuration ordering.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, Leox LXT-010H-D

1100 down / 450 up, Bufferbloat A+

Today at 05:59:07 PM #2 Last Edit: Today at 06:01:23 PM by Monviech (Cedrik)
You can add a sequence to the routes already and theyre rendered in that order. :)

Its also needed for other things so I added it almost at the beginning of that feature.

https://github.com/opnsense/plugins/blob/261c32ef7196199a7bc0224bc4df7b9a02eb1d57/www/caddy/src/opnsense/mvc/app/controllers/OPNsense/Caddy/forms/dialogLayer4.xml#L17
Hardware:
DEC740

Thank you, that solves it!

(I have to admit that I even saw this PR, but didn't realize from the problem description that it fits to my issue.)