Hi :)
I did install and run crowdsec since a while, did activate prometheus metrics and scrape them, then I didn't look at this so much
I did purchase enterprise plan, so, I did believe that subscribed block-lists prevented alerts to occur
After upgrading to 25.7 I did see some metrics (due to reboot of opnsense and crowdsec) then I did dive deeper in crowdsec
I looks like everything works fine for a moment then metrics show me that there is no more parsed line
I did check from cscli parsers inspect crowdsecurity/nginx-logs and it looks like metrics are right, running the command multiple times doesn't show hits/parsed number increasing at all, and files <nginx>.access|error.log are still logging new lines
I don't know if this is relative to 25.7 or not because as said, I didn't look at crowdsec deep enough until few days
I can see some warnings like level=warning msg="file reader died : Failed to detect creation of /var/log/nginx/permanentban.access.log: \"/var/log/nginx/latest.log\": open /var/log/nginx/latest.log: not a directory" tail=/var/log/nginx/permanentban.access.log type=filebut I did exclude /var/log/nginx/latest.log from config
:~ # cat /usr/local/etc/crowdsec/acquis.yaml
filenames:
- /var/log/nginx/*.error.log
- /var/log/nginx/*.access.log
#this is not a syslog log, indicate which kind of logs it is
labels:
type: nginx
---
filenames:
- /var/log/auth.log
- /var/log/syslog
labels:
type: syslog
---
filenames:
- /var/log/httpd-access.log
- /var/log/httpd-error.log
labels:
type: apache2
So do someone can see crowdsec stop parsing nginx log files after some times (like one hour) ?
Well, after reading logs, it appears that the problem come from /var/log/nginx/permanentban.access.log that is deleted/recreated to often and crowdsec seems to wait for it and ... finally the reader died for the nginx directory
I did add :
exclude_regexps:
- permanentban*
in /usr/local/etc/crowdsec/acquis.yaml
and it seems that crowdsec is now stable ... wait and see