I've written on crowdsec discourse about Caddy JSON float‑ts isnt parsed on OPNsense leading to false http‑crawl‑non_statics bans (https://discourse.crowdsec.net/t/caddy-json-float-ts-isn-t-parsed-on-opnsense-leading-to-false-http-crawl-non-statics-bans/2512/2)
I've also read in the Caddy logs that the json time format can be changed (https://caddyserver.com/docs/json/logging/logs/encoder/json/time_format/)
Recognized values are: unix_seconds_float, unix_milli_float, unix_nano, iso8601, rfc3339, rfc3339_nano, wall, wall_milli, wall_nano, common_log. The value may also be custom format per the Go time package layout specification.
and if so then the default crowdsec parsers should work e.g. with rfc3339, rfc3339_nano, iso8601 etc
Does anyone know where that may be set globally? Thank you for any help in advance it is much appreciated.
https://github.com/opnsense/plugins/blob/8d2972a4ac7045ad5ec403adc736da817cc540b9/www/caddy/src/opnsense/service/templates/OPNsense/Caddy/Caddyfile#L56
Though keep in mind that syslog-ng uses this timestamp too.
Thank you. I can see my current syslog time_format is rfc3339 and in the caddy own logs it shows that timestamp format.
However individual domain http logs begin unix time:
{"level":"info","ts":1744966843.7126594,"....
I tried to create /usr/local/etc/caddy/caddy.d/custom-time.global with content:
log {
format json {
time_format rfc3339
}
}
However I got the error:
Error: adapting config using caddyfile: parsing caddyfile tokens for 'log': duplicate global log option for: default, at /usr/local/etc/caddy/caddy.d/custom-time.global:1 import chain ['/usr/local/etc/caddy/Caddyfile:23 (import)']
From what I understand, this happens because a log block is already defined globally, and Caddy only supports one global log block.
In the JSON configuration diagnostics, I also noticed that the per-site access logs (log0 to log3) are excluded from the global logger:
"logging": {
"logs": {
"default": {
"encoder": {
"format": "json",
"time_format": "rfc3339"
},
"exclude": [
"http.log.access.log0",
"http.log.access.log1",
"http.log.access.log2",
"http.log.access.log3"
Each of those access logs has its own writer block, but they don't inherit the global log format and therfore the unix timestamp format still being used.
Or have I approached this incorrectly and I should be running crowdsec on opnsense only for the opnsense protection and running secondary instances of crowdsec on each webserver and monitoring their own proxy logs and reporting back to the main crowdsec?
My Questions:
Is there a way to apply the same json + rfc3339 formatting to all domain access logs without defining it separately for each domain UUID?
Alternatively, am I approaching this the wrong way and should CrowdSec on OPNsense only protect OPNsense itself, and I should run secondary CrowdSec instances on each VM/webserver to monitor their local proxy access logs and report back to the main instance?
My hope was to have a single CrowdSec instance on OPNsense that could monitor all webserver traffic routed through it via Caddy, and apply bans appropriately. But I'm beginning to think full log visibility might require local logging and parsing on each web server.
Appreciate any guidance or confirmation either way.
Sorry I cannot help much here.
If crowdsec responds and says the unixtime logformat is wrong, open a ticket in github opnsense plugins so we can fix it.