Update with crowdsec Plugin

[gurbas]

Hi
OPNsense Version: 24.1.9_4

Autoupdate hang on 2 of 3 installations.

It is waiting for the process to stop - simple rebooting was on the first machine no solution.
Had to attach a monitor.
... try to kill the preocess manually now


anybody else?

[34/41] Extracting crowdsec-1.6.2_2: .......... done
crowdsec is running as pid 23724.
Stopping crowdsec.
Waiting for PIDS: 23724.
Waiting for PIDS: 24342

[gurbas]

After rebooting - that worked this time flawless - some missing Updates needed to be finished.
After a reboot everythings working as expected again.

[starkstromkonsument]

Hi,

I have exactly the same problem on 24.7. I don't know on which version it started. Running

Code Select Expand

kill -9 <PID of hung crowdsec process> via ssh solves the hanging and the update / reboot completes as intended.

It has been discussed before: https://forum.opnsense.org/index.php?topic=34435.msg166789#msg166789 but I can't find any solution here in the forum (seraching for crowdsec stuck or crowdsec hungs or similar) nor in the github issues for plugins.

Does anybody have a hint or link?

[starkstromkonsument]

This is from my System: Log Files: Backend

2024-08-26T22:17:33   Error   configd.py   Timeout (120) executing : crowdsec stop   
2024-08-26T22:16:44   Error   configd.py   Timeout (120) executing : crowdsec stop   
2024-08-26T22:16:44   Error   configd.py   Timeout (120) executing : service stop 'crowdsec' ''

[roens]

I also have faced this for recent updates requiring reboot. I too need to log in via ssh to kill the Crowdsec process by PID.

Pretty dysfunctional and frustrating.

Related:
- https://forum.opnsense.org/index.php?topic=43067.0
- https://forum.opnsense.org/index.php?topic=34355.0

Edit: As this has been a persistent issue for me, and I see others here have run into it, I've created a GH issue: https://github.com/opnsense/plugins/issues/4262

[ObraDinn]

Fyi, same issue here...: updating from OPNsense 24.7.5_3-amd64

Code Select Expand

....
[40/43] Extracting crowdsec-1.6.3_2: .......... done
crowdsec is running as pid 76096.
Stopping crowdsec.
Waiting for PIDS: 76096.
Waiting for PIDS: 86798

The PID command is:

Code Select Expand

/usr/local/bin/crowdsec -c /usr/local/etc/crowdsec/config.yaml{crowdsec}

[Patrick M. Hausen]

killall -9 crowdsec

[36thchamber]

yep, crowdsec implementation is the menace of the year and blocking the router for months. blocked upgrades, and even UPS events succesfully.
even in 24.7.7, can't stop it, it also keeps trying its own port which is already reserved by itself every 10sec:

Code Select Expand

local API server stopped with error: listening on 127.0.0.1:8080: listen tcp 127.0.0.1:8080: bind: address already in use

and it does nonstop internet activity during that, fetching from api.crowdsec.net nonstop. hillariously, after killing it, and starting from scratch, it killed the OS:

Code Select Expand


ps aux|grep crowdsec
-
sockstat -sSUivl|grep 8080
-
configctl crowdsec start
OK
root@opnsense:/]$                                                                               
*** FINAL System shutdown message from root@opnsense ***               

System going down IMMEDIATELY

[Patrick M. Hausen]

That was the "hung" shutdown process in the background that was finally able to continue after you killed the crowdsec process.

As far as I know the issue is supposed to be fixed now. And yes, it was bad, especially so if your system is in a remote location.

[36thchamber]

Stuck again in update from 24.7.7 to 24.7.8, wish that wait_for_pids function had a timeout. TERM signal doesn't work.

[FreeMinded]

I can confirm the issue. It happend to me again while updating from 24.7.5 to 24.7.8
The first part of the update ran fine and firewall rebooted, it got stuck while updating the plugins
Trying to stop or disable crowdsec from the GUI doesn't do it. had to use

Code Select Expand

kill -9

[FreeMinded]

For my understanding: Is this issue something that has to be solved on the crowdsec side or on the OPNsense side?
Is there an open issue on either side? I could not find any yet.

[36thchamber]

For my understanding: Is this issue something that has to be solved on the crowdsec side or on the OPNsense side?
Is there an open issue on either side? I could not find any yet.

#4262 on GIT. still active in 24.7.12_2. Even Adguard can't restart. The issue is nothing is killed, OPNsense scripts are hoping processes will terminate, and they will not.

[dinguz]

I also experienced this issue and reported it with the plugin maintainer. An updated plugin should be released soon, although I'm not sure for which OPNsense versions it will be released.