Messages

Hi,

can you test with the 1.4.3 package? It has not landed in opnsense ports yet: https://www.freshports.org/security/crowdsec/

# pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/crowdsec-1.4.3.pkg

Hi!

While Linux and Windows versions are released automatically, the FreeBSD one must follow a separate review/approval process (it's an official distribution package), so it lags a few days to get to the freebsd ports + a few days to land in the opnsense repository.

See the related ticket at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267808
and the package status page at https://www.freshports.org/security/crowdsec

If you need it now, I can send you a binary for testing.

Hello nzkiwi68!

The blocklist "community" updates come from the cloud service, yes. Then there are local attacks which are detected, flagged and banned in real time (depending on the rules of the scenario, a community decision may not be required to trigger a ban). The crowdsec daemon receives and writes the list of IPs to the local database (see "cscli decisions list -a")

There is no need to reload the filter service since there are a couple of dynamic PF tables (for ipv4 and ipv6) which are updated by the crowdsec-firewall-bouncer daemon.

What have you upgraded exactly? And which service is not coming up?

If it's crowdsec, you can check /var/log/crowdsec/crowdsec.log, crowdsec-firewall-bouncer.log and post the errors you see (grep -v level=info may help)

Hi!

In the "Plain View" of the logs, you can see the full lines:

2022-08-30T09:33:41   Informational   filterlog   57,,,blocked by crowdsec,vtnet0,match,block,in,4,0x0,,64,36851,0,DF,17,udp,72,192.168.122.1,192.168.122.255,57621,57621,52

I don't know why the live logs don't show the details anymore. I'm pretty sure it worked when I wrote it but I don't have a 22.1 version to check if there is any difference, or if it's something I did.

I have to check, if someone has ideas, my code to register the rules is the following

    add_alias_if_not_exist('crowdsec_blacklists', 'CrowdSec (IPv4)', 'IPv4');

    // https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/library/OPNsense/Firewall/FilterRule.php

    $fw->registerFilterRule(
        1, /* priority */
        array(
            'ipprotocol' => 'inet',
            'descr'      => 'CrowdSec (IPv4)',
            'from'       => '$crowdsec_blacklists',     # $ to reference an alias
            'type'       => 'block',
            'log'        => $rules_log_enabled,
            'tag'        => $rules_tag,
            'label'      => 'blocked by crowdsec',
            'quick'      => true
        ),
        null
    );



(and again for ipv6)

Hello PIv0!

We have not written anything about HA with opnsense yet because we have not tested that configuration. Do you have an external LAPI or do you need a LAPI on each firewall instance?

I know some of our users have LAPI HA (without opnsense) and I can reach out, if it can be applied to this case we can consider it for a next release.

Other than that, most of the documentation at https://docs.crowdsec.net/ also applies to the opnsense version.

Hi!

The agent (which is half of crowdsec) is trying to connect to the LAPI (the other half) and failing.

When you removed the db without also removing local_api_credentials.yaml, you caused a password mismatch.


I suppose you want agent and LAPI both on the same machine.

In this case, verify that:

- you have os-crowdsec 1.0 and not os-crowdsec-devel
- the settings have the following default values:

Y - Enable CrowdSec (IDS)
Y - Enable LAPI
Y - Enable Firewall Bouncer (IPS)
N - Manual LAPI configuration

LAPI listen address: 127.0.0.1
LAPI listen port: 8080

You can apply these and see if it works.

If it doesn't and you can reinstall: remove the plugin, check that the packages crowdsec and crowdsec-firewall-bouncer have been removed, then delete by hand /var/db/crowdsec, /usr/local/etc/crowdsec before reinstalling again.

If you don't want to reinstall you can remove the machine, remove login and password from /usr/local/etc/crowdsec/local_api_credentials.yaml and restart the services, that should fix it but I have not tried.

If your LAPI is on a different machine, you should follow the directions in https://docs.crowdsec.net/docs/next/getting_started/install_crowdsec_opnsense


Let me know..

Hello!

I suppose you are running the 1.0 version (crowdsec 1.3.4), with opnsense 22.7

> Enable Firewall Bouncer (IPS)
> When this is enabled I get no alerts for blocks in the firewall logs.

You mean you don't see anything in /ui/diagnostics/log/core/filter ?

or with

Code Select Expand

# cat /var/log/filter/latest.log  | grep 'blocked by crowdsec'

Is the process "crowdsec-firewall-bouncer" running?

What's in /var/log/crowdsec/crowdsec-firewall-bouncer.log ? If you enable verbose debug in the settings tab, you should see the calls to pfctl there too.

Another thing to try (for ipv4):

Code Select Expand

# pfctl -t crowdsec_blacklists -T show

mmetc, I went and removed the three crowdsec files then moved over to the developer settings.  Reinstalled  Crowdsec using the 0.2 plugin and same problem.  Overview shows a Red X in the Service Status and a Green Checkmark in the Firewall Bouncer.  Pulled the crowdsec.log and attached are the contents:

It seems like your sqlite database was manually removed ( /var/db/crowdsec/data/crowdsec.db ) without removing /usr/local/etc/crowdsec at the same time.  So it looks for an agent, referenced in the configuration, that does not exist in the db.

You can remove the packages, remove these directories as well, and reinstall. Or see what "cscli machines list" says and fix the configuration

As an aside, moving to the Opnsense developer trashed Zenarmor with a fatal error concerning Phalcon not found so beware anyone taking this route.

Of course.

No joy in my attempt at a 3rd installation.  I removed all three packages; Made sure the Aliases have been deleted from the Firewall.  Then reinstalled in the files in the order specified.  Checked the blocks in the setting tab to get it started.  In the Overview there is a Red X in the Crowdsec service and a Green Check Mark in the Bouncer service. 

I don't get it.  It worked fine before this last update....

Hi, I'm the author of the plugin.

You can tail /var/log/crowdsec/crowdsec.log to see why the service dies.

If you don't switch to the development repository, which I guess is your case, the old version should still work, even if you upgraded the binaries to 1.3.3 or 1.3.4. The plugin itself will be updated when it hits the "community" repository.

If you want to use the version in the -devel repository, you can:

- activate the development repository in Firmware -> Settings (nb this updates way more than crowdsec)
- remove the three packages if you have installed them by hand (os-crowdsec, crowdsec, crowdsec-firewall-bouncer). It is better to remove them by hand to avoid any conflict or version mismatch and allow them to be removed at once when you eventually remove the plugin. Otherwise the plugin is removed and the binaries are not.
- Install os-crowdsec-devel 0.2.

It worked for me but if a service is down after the upgrade, I'd like to know more from the log files. No manual intervention to the system/configuration files is necessary.

Hi phantomsfbw!

I will be able to test later, but it seems -- since you talk about two packages -- that you reinstalled crowdsec and the firewall bouncer, not the opnsense plugin.

I guess what happened is this: you received a new version of crowdsec 1.3.3 from the opnsense repository, which is good, and it reinstalled over the version you manually installed (likely 1.3.2 - I never packaged the 1.3.3 binary). But the update process has removed the plugin, which - surprise! - is a third package.

So you should take it from https://github.com/crowdsecurity/opnsense-plugin-crowdsec/releases/download/v0.1/opnsense_22.1-freebsd_13-oscrowdsec_0.1.tar and run "pkg install os-crowdsec-0.1.txz". Go in settings, check and save.

Leave /etc/rc.conf empty, as you find it. The good stuff is in /etc/rc.conf.d/ anyway and is automatically generated from templates.
In general, messages from package installs are meant in the context of vanilla freebsd, not opnsense.

Whether you run crowdsec 1.3.2 or 1.3.3 should be irrelevant. Version 1.3.4 is in the pipes and the plugin has been merged for the next release as well, so there will be no need to manually install anything.

Thank you, I somehow lost the notification of your reply and was coming back here to say the same thing.

Now I have a "configctl crowdsec remove-alias" event that I call from +PRE_DEINSTALL.pre and it works well.

Thank you for your response, I'll create a ticket for this issue.

I can certainly tell the users to remove the aliases by hand.

As a temporary measure, I was playing with this script in +PRE_DEINSTALL.pre (or POST) but it doesn't work.

#!/bin/sh

/usr/local/bin/php << 'EOT'
<?php

@include_once("config.inc");
@include_once("certs.inc");
@include_once("util.inc");

use OPNsense\Firewall\Alias;
use OPNsense\Core\Config;

function removeAlias($name)
{
    $model = new Alias();
    foreach ($model->aliases->alias->iterateItems() as $index => $alias) {
        if (strval($alias->name) == $name) {
            if ($model->aliases->alias->del($index)) {
                $model->serializeToConfig();
            }
        }
    }
}

removeAlias('crowdsec_blacklists');
removeAlias('crowdsec6_blacklists');
EOT

For reference. A "pfctl -k" after adding IPs does the job.

But I could not keep the anchor and had to use a table associated with an Alias.
The same code would blocks packets with an Alias table, do nothing with an anchored table.

I'm not sure why because other plugins use anchors too, but I had my fix.

thanks

Hello!

In my crowdsec plugin, still unpublished, I create my stuff in plugins.inc.d/crowdsec.inc, then I call "configctl filter reload" at the time of installation. Not sure it's the best way, but it seems to work as I want.

My question is: can I remove the Alias objects when my plugin is uninstalled? I see some

+POST_DEINSTALL.post

scripts but should I call my php code from there? It's fine if the Alias is removed and recreated when the plugin is upgraded.

Is there a plugin that does a similar clean up?

Thanks

[...]
function add_alias_if_not_exist($name, $description, $proto) {
    $model = new OPNsense\Firewall\Alias();
    foreach ($model->aliases->alias->iterateItems() as $alias) {
        if ((string)$alias->name == $name) {
            return;
        }
    }

    $new_alias = $model->aliases->alias->Add();
    $new_alias->name = $name;
    $new_alias->description = $description;
    $new_alias->proto = $proto;
    $new_alias->type = 'external';
    $model->serializeToConfig();
    Config::getInstance()->save();
}

function crowdsec_firewall(\OPNsense\Firewall\Plugin $fw)
{
    if (!bouncer_enabled()) {
        return;
    }

    add_alias_if_not_exist('crowdsec_blacklists', 'CrowdSec (IPv4)', 'IPv4');

    $fw->registerFilterRule(
        1, /* priority */
        array(
            'ipprotocol'     => 'inet',
            'descr'          => 'CrowdSec (IPv4)',
            'from'           => '$crowdsec_blacklists',     # $ to reference an alias
            'type'           => 'block',
            'quick'          => true
        ),
        null
    );

    add_alias_if_not_exist('crowdsec6_blacklists', 'CrowdSec (IPv6)', 'IPv6');

    $fw->registerFilterRule(
        1, /* priority */
        array(
            'ipprotocol'     => 'inet6',
            'descr'          => 'CrowdSec (IPv6)',
            'from'           => '$crowdsec6_blacklists',    # $ to reference an alias
            'type'           => 'block',
            'quick'          => true
        ),
        null
    );
}
[...]