Messages

1

24.7 Production Series / Re: Monit alert - Resource limit matched

« on: October 08, 2024, 03:57:43 pm »

You can find the official crowdsec thread on this issue here:

https://discourse.crowdsec.net/t/bug-opnsense-24-7-5-crowdsec-1-6-3/2057

2

24.7 Production Series / Re: Monit alert - Resource limit matched

« on: October 08, 2024, 03:55:27 pm »

I have executed the command "killall crowdsec".
12834 of 13158 are processes by the name "notification-*".

How can I stop these processes?

"kill 12834" and keep the most recent.

Why does this problem occur?

tl;dr my fault, longer version: each freebsd package does service management in a slightly different way: start, stop, restart if error but not too often, reload configuration, coordinate process groups... there is no unified way to express the application's needs, like the - admittedly not universally popular - systemd system in linux. Which means it requires more scripts to manage corner cases, and more room for errors.

3

24.7 Production Series / Re: Monit alert - Resource limit matched

« on: October 08, 2024, 02:41:17 pm »

Hi,

could you test this

# fetch -o /usr/local/etc/rc.d/crowdsec https://github.com/crowdsecurity/plugins/releases/download/crowdsec-1.6.3-2-hotfix/crowdsec

and try start/stop.

Thanks

Do I have to use an additional command to install the hotfix?
I suspect that the update did not work. (Screenshots attached)

No it's ok. The fetch command overwrites a script without installing a new package version. Now if you click start/stop from the UI it should just work.

Thank you for the information.

I have noticed that the service can be stopped via the GUI. (Visible because service status is deactivated in the crowdsec overview)
However, the service is displayed as active in the dashboard and in the overview of services.

You have orphan crowdsec processes and possibly notification plugins.

"killall crowdsec" and check if there are processes that go by the name "notification-*"

4

24.7 Production Series / Re: Monit alert - Resource limit matched

« on: October 08, 2024, 09:13:08 am »

Hi,

could you test this

# fetch -o /usr/local/etc/rc.d/crowdsec https://github.com/crowdsecurity/plugins/releases/download/crowdsec-1.6.3-2-hotfix/crowdsec

and try start/stop.

Thanks

Do I have to use an additional command to install the hotfix?
I suspect that the update did not work. (Screenshots attached)

No it's ok. The fetch command overwrites a script without installing a new package version. Now if you click start/stop from the UI it should just work.

5

24.7 Production Series / Re: Monit alert - Resource limit matched

« on: October 07, 2024, 03:34:24 pm »

Hi,

could you test this

# fetch -o /usr/local/etc/rc.d/crowdsec https://github.com/crowdsecurity/plugins/releases/download/crowdsec-1.6.3-2-hotfix/crowdsec

and try start/stop.

Thanks

6

24.7 Production Series / Re: Monit alert - Resource limit matched

« on: October 07, 2024, 09:44:55 am »

Hi, crowdsec maintainer here

First thing, the daemon manager had an issue and ignored the INT signal sometimes, in this case the upgrade tries to stop the service and fails.

root    40599  4.0  6.8 1390784 104344  -  S    09:38   0:01.64 /usr/local/bin/crowdsec -c /usr/local/etc/crowdsec/config.yaml
root    40515  0.0  0.1   12736   2164  -  Ss   09:38   0:00.00 daemon: crowdsec[40599] (daemon)

terminate the second process (kill -9 40515) and upgrade to 1.6.3. This changes the script to send a "stronger" signal to stop the process.

Now to understand why it happened, it would be helpful if you could run "cscli support dump" and send the result to support@crowdsec.net. Let us know if, after the upgrade, you still think crowdsec uses too much cpu or ram. It's not a lightweight process but it should not trigger monitoring.

Thanks

7

24.7 Production Series / Re: click on menu item: system-firmware-status -> OPNsense reboots immediately

« on: October 02, 2024, 09:29:36 pm »

Hi, I'm the plugin maintainer

This can happen with 1.6.2 when crowdsec is misbehaving for other reasons (possibly misconfigured, port not available etc) then the daemon manager won't stop it correctly because it's stuck in a restart loop. The opnsense upgrade should install 1.6.3 which fixed the issue by using a "stronger" signal to terminate the daemon manager.

I suggest kill -9, or even the broader

```
# kill -9 `ps xw | grep crowdsec | grep -v grep | awk '{print $1'}`
```

then update to 1.6.3, but should happen with the opnsense upgrade


If there is an underlying issue preventing crowdsec from working correctly, the most "complete" way to report it is via "cscli support dump" and email the resulting file to support@crowdsec.net. This includes logs and part of the configuration so it saves us time asking for details.

8

Intrusion Detection and Prevention / Re: crowdsec firewall bouncer does not start - pfctl crowdsec-blacklists not exist

« on: June 05, 2024, 04:49:42 pm »

Now indeed in OPNSense -> Services -> CrowdSec -> Overview it's indeed better:
Service status: crowdsec [tick / success] - firewall bouncer [tick / success]

But it's still unclear to me why this happens on a stock Install ... and for how long it would even work .

If there's no error in the reconfigure event, it should keep working. I don't see why it failed the first time.

9

Intrusion Detection and Prevention / Re: crowdsec firewall bouncer does not start - pfctl crowdsec-blacklists not exist

« on: June 05, 2024, 04:47:27 pm »

ha ha no. I meant discord. You missed my subsequent post
Frankly no idea which one is meant to be the official place for support requests. I hope it is Github.

Discord or Reddit are good for interactive or non-technical support, GitHub for better follow up.

10

Intrusion Detection and Prevention / Re: crowdsec firewall bouncer does not start - pfctl crowdsec-blacklists not exist

« on: June 05, 2024, 01:52:20 pm »

sometimes the crowdsec people respond here but best to report directly. They seem to be active on their online thingie that I can't remember what is called.

Maybe you mean GitHub?

11

Intrusion Detection and Prevention / Re: crowdsec firewall bouncer does not start - pfctl crowdsec-blacklists not exist

« on: June 05, 2024, 01:45:29 pm »

I installed (or rather attempted to) Crowdsec on the latest OPNSense Release (with all Updates applied: OPNsense 24.1.8-amd64, FreeBSD 13.2-RELEASE-p11, OpenSSL 3.0.13) according to https://docs.crowdsec.net/docs/getting_started/install_crowdsec_opnsense/.

I also enrolled it to the Crowdsec Console (from SSH-ing into my OPNSense Instance).

However, while the Crowdsec Service appears to work correctly, the Firewall Bouncer dies within a second or so after attempting to be started.

OPNSense -> Services -> CrowdSec -> Overview
Service status: crowdsec [tick / success] - firewall bouncer [cross / fail]

Output of `cscli version`:

Code: [Select]

2024/06/04 17:00:55 version: v1.6.1-freebsd-0746e0c0
2024/06/04 17:00:55 Codename: alphaga
2024/06/04 17:00:55 BuildDate: 2024-05-28_00:23:25
2024/06/04 17:00:55 GoVersion: 1.21.10
2024/06/04 17:00:55 Platform: freebsd
2024/06/04 17:00:55 libre2: C++
2024/06/04 17:00:55 Constraint_parser: >= 1.0, <= 3.0
2024/06/04 17:00:55 Constraint_scenario: >= 1.0, <= 3.0
2024/06/04 17:00:55 Constraint_api: v1
2024/06/04 17:00:55 Constraint_acquis: >= 1.0, < 2.0
According to the logs, it seems one Blacklist doesn't exist. Am I supposed to create it manually (it wasn't in the Tutorial), and if so, how ?

OPNSense -> Firewall -> Aliases show that "crowdsec_blacklists" and "crowdsec6_blacklists" exists.
Note the "_" (underscore) instead of the "-" (dash) which pfctl complains in the logs below.

Output of `cat /var/log/crowdsec-firewall-bouncer.log`

Code: [Select]

time="04-06-2024 16:22:55" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:22:55" level=info msg="backend type : pf"
time="04-06-2024 16:22:55" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:22:55" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:22:55" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:22:55" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:34:42" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:34:42" level=info msg="backend type : pf"
time="04-06-2024 16:34:42" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:34:42" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:34:42" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:34:42" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:50:43" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:50:43" level=info msg="backend type : pf"
time="04-06-2024 16:50:43" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:50:43" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:50:43" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:50:43" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:50:47" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:50:47" level=info msg="backend type : pf"
time="04-06-2024 16:50:47" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:50:47" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:50:47" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:50:47" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:50:50" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:50:50" level=info msg="backend type : pf"
time="04-06-2024 16:50:50" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:50:50" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:50:50" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:50:50" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:54:03" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:54:03" level=info msg="backend type : pf"
time="04-06-2024 16:54:03" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:54:03" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:54:03" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:54:03" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:55:04" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:55:04" level=info msg="backend type : pf"
time="04-06-2024 16:55:04" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:55:04" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:55:04" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:55:04" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:55:06" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:55:06" level=info msg="backend type : pf"
time="04-06-2024 16:55:06" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:55:06" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:55:06" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:55:06" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"
time="04-06-2024 16:55:06" level=info msg="Starting crowdsec-firewall-bouncer v0.0.28-freebsd-af6e7e2"
time="04-06-2024 16:55:06" level=info msg="backend type : pf"
time="04-06-2024 16:55:06" level=info msg="pf table clean-up: /sbin/pfctl -t crowdsec-blacklists -T flush"
time="04-06-2024 16:55:06" level=error msg="Error while flushing table (/sbin/pfctl -t crowdsec-blacklists -T flush): exit status 255 --> pfctl: Table does not exist.\n"
time="04-06-2024 16:55:06" level=info msg="Checking pf table: crowdsec-blacklists"
time="04-06-2024 16:55:06" level=fatal msg="pf init failed: table crowdsec-blacklists doesn't exist"

The plugin should configure /usr/local/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

to use the _ instead of - which is the default value, but not allowed by opnsense.

Can you please run

# configctl crowdsec reconfigure

# tail -f /var/log/configd/latest.log

and see if there's any error?

thanks

12

General Discussion / Re: Upgrade to -> 22.7 Crowdsec not working.

« on: May 24, 2024, 11:07:11 am »

If you don't want to reinstall you can remove the machine, remove login and password from /usr/local/etc/crowdsec/local_api_credentials.yaml and restart the services, that should fix it but I have not tried.

this way didn't work, but the removal of db and config folder worked thanks.

this plugin and ntopng are high maintenance, and sort of unreliable. crowdsec GUI also always lies everything is OK. the alias is getting empty, i have a scheduled task to restart crowdsec every few hours. i thought i was out of woods, and stopped the months long stressful watching, but today i noticed the blocklist is again empty. checked the logs, and it didn't load for 2months! i don't know why, but i've also got the same issue here.

Hi, if you are running the latest version could you please send to support@crowdsec.net:

 - the result of "cscli support dump"
 - the content of /var/log/crowdsec
 - the output of "sed -n '/<crowdsec>/,/<\/crowdsec>/p' /conf/config.xml"

I'd like to get to the bottom of this, thanks!

13

24.1 Legacy Series / Re: After update to 24.1.7Crowdsec will not start with LAPI disabled

« on: May 18, 2024, 09:42:30 am »

Thanks! Quick fix and PR in

https://github.com/opnsense/plugins/issues/3985#issuecomment-2118451676

14

24.1 Legacy Series / Re: Crowdsec Daemon is stopping at 1am (sometimes)

« on: March 19, 2024, 02:28:09 pm »

Hello,

Thanks for sending logs and configurations, we fixed some issue for the upcoming 1.6.1 and are looking at other possible causes.

In the meantime, we have a version of the base crowdsec package that restarts the service correctly when it fails.

You can find it at https://github.com/crowdsecurity/plugins/releases/tag/crowdsec-1.6.0_3

Let us know if it helps and thanks for testing,

Marco

15

24.1 Legacy Series / Re: Crowdsec Daemon is stopping at 1am (sometimes)

« on: March 15, 2024, 04:10:48 pm »

Yes crowdsec would inappropriately raise an error if a watched file disappears immediately after the initial directory scan
This will be corrected for 1.6.1, but I'm not sure how often it occurs.
More generally, a process exit by crowdsec could be due to CAPI being unavailable for a long time or other issues.

On the linux package any transient exit/crash is not a problem, expect for the possible underlying bug, since the process is restarted immediately by systemd (or docker). For freebsd there is no - afaik - general consensus on how to restart crashed processes.

Monit is a good solution but it's not available on freebsd by default or in pfsense. I tried simply adding a restart option to the sbin/daemon wrapper, it's not working as expected but I'd prefer the solution should be the same for the three platforms.

If someone is using monit to restart crowdsec, can you share that part of configuration?

Thanks