OPNsense Forum

Archive => 22.1 Legacy Series => Topic started by: phantomsfbw on May 11, 2022, 03:36:35 AM

Title: Crowdsec
Post by: phantomsfbw on May 11, 2022, 03:36:35 AM
i was on 22.1.6 and had installed Crowdsec manually and it seemed to work fine.  Today i installed the new 22.1.7 and Crowdsec quit working. OS-crowdsec shows it has been orphaned in the plugins section.  I deleted the orphan and tried to reinstall the new Crowdsec packages, there are two and I get this error:
***GOT REQUEST TO REINSTALL***
Currently running OPNsense 22.1.7 (amd64/OpenSSL) at Tue May 10 21:31:04 EDT 2022
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
pkg-static: https://updates.sunnyvalley.io/opnsense/FreeBSD:13:amd64/22.1/OpenSSL/latest/packagesite.pkg: Not Found
SunnyValley repository is up to date.
All repositories are up to date.

No packages are required to be fetched.
Integrity check was successful.
crowdsec-1.3.3: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
pkg-static: https://updates.sunnyvalley.io/opnsense/FreeBSD:13:amd64/22.1/OpenSSL/latest/packagesite.pkg: Not Found
SunnyValley repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
   crowdsec-1.3.3 [OPNsense]

Number of packages to be reinstalled: 1
[1/1] Reinstalling crowdsec-1.3.3...
[1/1] Extracting crowdsec-1.3.3: .......... done
Cannot 'status' crowdsec. Set crowdsec_enable to YES in /etc/rc.conf or use 'onestatus' instead of 'status'.
Cannot 'stop' crowdsec. Set crowdsec_enable to YES in /etc/rc.conf or use 'onestop' instead of 'stop'.
Cannot 'start' crowdsec. Set crowdsec_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.
You may need to manually remove /usr/local/etc/crowdsec/local_api_credentials.yaml if it is no longer needed.
You may need to manually remove /usr/local/etc/crowdsec/online_api_credentials.yaml if it is no longer needed.
You may need to manually remove /usr/local/etc/crowdsec/config.yaml if it is no longer needed.
Checking integrity... done (0 conflicting)
Nothing to do.
***DONE***

I found the /etc/rc.conf file empty when I went to edit.  I was able to manually remove the recommended files, but that did not change anything.  How do I get Crowdsec back up and running?
Title: Re: Crowdsec
Post by: mmetc on May 11, 2022, 04:22:33 PM
Hi phantomsfbw!

I will be able to test later, but it seems -- since you talk about two packages -- that you reinstalled crowdsec and the firewall bouncer, not the opnsense plugin.

I guess what happened is this: you received a new version of crowdsec 1.3.3 from the opnsense repository, which is good, and it reinstalled over the version you manually installed (likely 1.3.2 - I never packaged the 1.3.3 binary). But the update process has removed the plugin, which - surprise! - is a third package.

So you should take it from https://github.com/crowdsecurity/opnsense-plugin-crowdsec/releases/download/v0.1/opnsense_22.1-freebsd_13-oscrowdsec_0.1.tar and run "pkg install os-crowdsec-0.1.txz". Go in settings, check and save.

Leave /etc/rc.conf empty, as you find it. The good stuff is in /etc/rc.conf.d/ anyway and is automatically generated from templates.
In general, messages from package installs are meant in the context of vanilla freebsd, not opnsense.

Whether you run crowdsec 1.3.2 or 1.3.3 should be irrelevant. Version 1.3.4 is in the pipes and the plugin has been merged for the next release as well, so there will be no need to manually install anything.
Title: Re: Crowdsec
Post by: phantomsfbw on May 11, 2022, 04:55:14 PM
mmetc thanks for the possible solution.  You are correct in your assessment of what happened.  While the new Crowdsec and the Bouncer were provided in the packages section, the third package, os-crowdsec was not.  I did try install the os-crowdsec 0.1 version, which does then show up in the menu, and you can enable etc., but Crowdsec attempts to start and then fails after about a minute or so.  The status goes from green to red.   
Title: Re: Crowdsec
Post by: hushcoden on May 11, 2022, 07:12:39 PM
Quote from: phantomsfbw on May 11, 2022, 04:55:14 PM
...but Crowdsec attempts to start and then fails after about a minute or so.  The status goes from green to red.
Same issue, did you fix it?

Ta.
Title: Re: Crowdsec
Post by: phantomsfbw on May 11, 2022, 07:17:58 PM
Ta, I have not at this time.
Title: Re: Crowdsec
Post by: phantomsfbw on May 12, 2022, 02:58:26 AM
Alcon, after updating to the Opnsense .1 update this evening I went and pulled the os-crowdsec file and installed.  Then I went in and installed the two Crowdsec packages and stared the service.  It stoll does work.  The os-crowd  plug in reports it is misconfigured-unknown repository.  Now have an IPV4 DHCPD service stop.  Looks like  I'm going back to scratch.
Title: Re: Crowdsec
Post by: phantomsfbw on May 12, 2022, 04:14:47 AM
Anyone else out there got Crowdsec working on the latest OPNsense release?
Title: Re: Crowdsec
Post by: RamSense on May 12, 2022, 05:58:34 PM
working fine overhere. see https://forum.opnsense.org/index.php?topic=20153.msg137420#msg137420 (https://forum.opnsense.org/index.php?topic=20153.msg137420#msg137420)

=========
But have now this problem:
I just updated opnsense to 22.1.7
and noticed:
Installed packages to be DOWNGRADED:
   crowdsec: 1.3.2 -> 1.2.3 [mimugmail]

now crowdsec does not start... Others having this also? Should I try to install the latest version?

Answer: Updated to crowdsec - os-crowdsec-0.1.txz - and it is up and running again.

========

I did pkg upgrade xxxx for all the 3 files and that got it working again.
Title: Re: Crowdsec
Post by: franco on May 13, 2022, 07:42:25 AM
Michael said he removed the packages from his end. The 22.1.7_1 hotfix also added the latest development plugin to OPNsense and this will install it without any third party mirror:

# pkg install os-crowdsec-devel


Cheers,
Franco
Title: Re: Crowdsec
Post by: RamSense on May 13, 2022, 08:07:31 AM
thnx! pkg install os-crowdsec-devel
and all is ok running os-crowdsec-devel-0.2
Title: Re: Crowdsec
Post by: abulafia on May 13, 2022, 10:38:30 AM
Quote from: RamSense on May 12, 2022, 05:58:34 PM
I did pkg upgrade xxxx for all the 3 files and that got it working again.
Somewhat similar here - I re-installed from CLI and updated (via GUI). pulled 1.3.3. working fine now on os-0.1.

Now did a
pkg remove os-crowdsec
pkg install os-crowdsec-devel


and crowdsec seems to be up and running. System->Firmware->Plugins now also reports 0.2 and shows notes for the plugin.
Title: Re: Crowdsec
Post by: yeraycito on May 13, 2022, 08:24:11 PM
os-crowdsec-devel not working
Title: Re: Crowdsec
Post by: yeraycito on May 13, 2022, 08:29:11 PM
Even if the service is activated in the Dashboard, it still does not work. And if it is deactivated and reactivated from Services - Crowdsec, sometimes it is activated and deactivated after a short time. However, the bouncer is always activated.
Title: Re: Crowdsec
Post by: phantomsfbw on May 13, 2022, 09:04:56 PM
Seeing same behavior.  Also captured this from log if it is relevant?

2022-05-13T15:02:00-04:00
configctl   error in configd communication Traceback (most recent call last): File "/usr/local/sbin/configctl", line 66, in exec_config_cmd line = sock.recv(65536).decode() socket.timeout: timed out
Title: Re: Crowdsec
Post by: phantomsfbw on May 17, 2022, 02:05:54 AM
No joy in my attempt at a 3rd installation.  I removed all three packages; Made sure the Aliases have been deleted from the Firewall.  Then reinstalled in the files in the order specified.  Checked the blocks in the setting tab to get it started.  In the Overview there is a Red X in the Crowdsec service and a Green Check Mark in the Bouncer service. 

I don't get it.  It worked fine before this last update....
Title: Re: Crowdsec
Post by: mmetc on May 19, 2022, 02:44:23 PM
Quote from: phantomsfbw on May 17, 2022, 02:05:54 AM
No joy in my attempt at a 3rd installation.  I removed all three packages; Made sure the Aliases have been deleted from the Firewall.  Then reinstalled in the files in the order specified.  Checked the blocks in the setting tab to get it started.  In the Overview there is a Red X in the Crowdsec service and a Green Check Mark in the Bouncer service. 

I don't get it.  It worked fine before this last update....

Hi, I'm the author of the plugin.

You can tail /var/log/crowdsec/crowdsec.log to see why the service dies.

If you don't switch to the development repository, which I guess is your case, the old version should still work, even if you upgraded the binaries to 1.3.3 or 1.3.4. The plugin itself will be updated when it hits the "community" repository.

If you want to use the version in the -devel repository, you can:

- activate the development repository in Firmware -> Settings (nb this updates way more than crowdsec)
- remove the three packages if you have installed them by hand (os-crowdsec, crowdsec, crowdsec-firewall-bouncer). It is better to remove them by hand to avoid any conflict or version mismatch and allow them to be removed at once when you eventually remove the plugin. Otherwise the plugin is removed and the binaries are not.
- Install os-crowdsec-devel 0.2.

It worked for me but if a service is down after the upgrade, I'd like to know more from the log files. No manual intervention to the system/configuration files is necessary.
Title: Re: Crowdsec
Post by: phantomsfbw on May 20, 2022, 01:16:56 AM
mmetc, I went and removed the three crowdsec files then moved over to the developer settings.  Reinstalled  Crowdsec using the 0.2 plugin and same problem.  Overview shows a Red X in the Service Status and a Green Checkmark in the Firewall Bouncer.  Pulled the crowdsec.log and attached are the contents:

As an aside, moving to the Opnsense developer trashed Zenarmor with a fatal error concerning Phalcon not found so beware anyone taking this route.


Title: Re: Crowdsec
Post by: mmetc on May 20, 2022, 12:14:32 PM
Quote from: phantomsfbw on May 20, 2022, 01:16:56 AM
mmetc, I went and removed the three crowdsec files then moved over to the developer settings.  Reinstalled  Crowdsec using the 0.2 plugin and same problem.  Overview shows a Red X in the Service Status and a Green Checkmark in the Firewall Bouncer.  Pulled the crowdsec.log and attached are the contents:

It seems like your sqlite database was manually removed ( /var/db/crowdsec/data/crowdsec.db ) without removing /usr/local/etc/crowdsec at the same time.  So it looks for an agent, referenced in the configuration, that does not exist in the db.

You can remove the packages, remove these directories as well, and reinstall. Or see what "cscli machines list" says and fix the configuration

Quote
As an aside, moving to the Opnsense developer trashed Zenarmor with a fatal error concerning Phalcon not found so beware anyone taking this route.

Of course.
Title: Re: Crowdsec
Post by: marvine on July 05, 2022, 08:26:16 PM
Quote from: RamSense on May 13, 2022, 08:07:31 AM
thnx! pkg install os-crowdsec-devel
and all is ok running os-crowdsec-devel-0.2

This solved for me the problem !