Crowdsec quits with new update

Started by dstrctdagain321, January 15, 2025, 08:25:55 PM

Previous topic - Next topic
Hi,

On my install, Crowdsec with 24.7.12 intermittently quits and restarts itself constantly. I am getting this error:

Script action failed with Command '/usr/local/bin/cscli alerts list -l 0 -o json' returned non-zero exit status 1. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/actions/script_output.py", line 78, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.11/subprocess.py", line 413, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/bin/cscli alerts list -l 0 -o json' returned non-zero exit status 1.
I removed the package, rebooted, reinstalled with the same issue. Would anyone else be experiencing this or have advice? Thank you :)

You can try reverting, and open an issue with Crowdsec about this.

# opnsense-revert crowdsec

Thank you!

Unfortunately it looks like the old Crowdsec package is no longer available to revert, it simply reinstalls the new one.

I will reach out to Crowdsec and see what I can do.

Must be something more particular to your setup.

Bouncer, scenarios, parsers, ... all perfectly up and running here.

I know that doesn't help much.

Have you tried invoking cscli manually? Then there's a Crowdsec Discord ...
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)

I'm guessing the same. Looking through their Discord now to see what I can find. Also, knowing your Crowdsec is working fine tells us a lot.

Are you using Crowdsec to parse Suricata logs by chance?

When reloading Crowdsec in the shell, it performs sanity check and I get:

time="2025-01-15T12:44:49-08:00" level=fatal msg="crowdsec init: while loading scenarios: scenario loading failed: unable to load alert context: compilation of 'match.matched_zones != nil ? match.matched_zones : ''' context value failed: unknown name match (1:1)\n | match.matched_zones != nil ? match.matched_zones : ''\n | ^"

My crowdsec is also working just fine after the update.  Nothing unusual in the logs.

After digging into logs, I was able to narrow it down to the Appsec collection. Removed it, and no more issues!

Same here, crowdsec didn't fully shutdown when stopping the service, requiring a kill -9 (this had been going on for some time). It wouldn't startup after that as well, spewing some rather cryptic messages (this started after upgrade to 24.7.12).
After removing the appsec collection the weird behaviour was gone.
In theory there is no difference between theory and practice. In practice there is.

Quote from: dinguz on January 16, 2025, 09:58:59 AMSame here, crowdsec didn't fully shutdown when stopping the service, requiring a kill -9 (this had been going on for some time). It wouldn't startup after that as well, spewing some rather cryptic messages (this started after upgrade to 24.7.12).
After removing the appsec collection the weird behaviour was gone.

Thank you. This resolved my issue.

Error: level=fatal msg="crowdsec init: while loading scenarios: scenario loading failed: unable to load alert context: compilation of 'match.msg != nil ? match.msg : ''' context value failed: unknown name match (1:1)\n | match.msg != nil ? match.msg : ''\n | ^"

The proper way to revert is to specify an OPNsense version ;)

# opnsense-revert -r 24.7.11 crowdsec

The crowdsec is restarting again for me too.
During my installation of the 24.7.12 .
Appsec collection how can I take it down?
Where can I find it?
Thank you.

By removing the collection, I managed to do it! :-)

Hi, I'm the plugin maintainer and was not able to reproduce the behavior.

If you had issues with the service start/stop during the package upgrade or at any other time, it would help if you run "cscli support dump" and send the output to support@crowdsec.net. It includes log files and part of the configuration (passwords removed ofc).

A look at that could also explain the initial errors of "cscli ... list", due to the service not running.
Thanks!

Quote from: mmetc on January 20, 2025, 04:52:59 PMIf you had issues with the service start/stop during the package upgrade or at any other time, it would help if you run "cscli support dump" and send the output to support@crowdsec.net. It includes log files and part of the configuration (passwords removed ofc).

Done, thanks in advance!
In theory there is no difference between theory and practice. In practice there is.

Had the same problem here. The new version of Crowdsec seems to have problems with appsec function. Fix by logging in to the console then revert to the last version.
opnsense-revert -r 24.7.11 crowdsec
Then fix the tainted collections with the following commands:
cscli collections upgrade --force crowdsecurity/appsec-generic-rules
cscli collections upgrade --force crowdsecurity/appsec-virtual-patching