OPNsense Forum

English Forums => Intrusion Detection and Prevention => Topic started by: JasMan on August 24, 2018, 02:32:06 pm

Title: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: JasMan on August 24, 2018, 02:32:06 pm
Hi,
I switched back from a PPoE Connection and therefore I can use IDS/IPS again.
I noticed that Suricata fails to start when I activate the rule list "abuse.ch/URLhaus". The following error is shown in the logs:

Aug 24 14:03:22    suricata: [100180] <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"URLhaus Known malware download URL detected"; flow:established,from_client; content:"GET"; http_method; content:"/url=http://wordpress.p364918.webspaceconfig.de/614tiscfz/com/us|26|data=02|01|rcorm1@jcp.com|ec2a6ed25318490bd27608d6077bf11e|9c0ac0b90217468aa4322649cd6ed297|0|0|636704626242706015|26|sdata=g3qlynktc59ma3fllqbbfs0uwnigsem1mwi/cdfotvu=|26|reserved=0"; http_uri; depth:250; isdataat:!1,relative; content:"na01.safelinks.protection.outlook.com"; http_host; depth:37; isdataat:!1,relative; metadata:created_at 2018_08_21; reference:url, urlhaus.abuse.ch/url/45633/; classtype:trojan-activity;sid:80908733; rev:1;)^M" from file /usr/local/etc/suricata/opnsense.rules/abuse.ch.urlhaus.rules at line 1595

I thougt "Okay, maybe there's a problem with the list. Wait some days and check again." But the issue still exist.

I've found only one other topic in this forum with nearly the same issue, but this was still unsolved. So I'm wondering why nobody else have this problem.
Is this an issue of Spamhouse or maybe OPNsense?

Jas
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: shred on August 28, 2018, 05:25:59 am
I'm seeing pretty much the same error but Suricata still starts. However, I've been having a lot of issues with OPNsense randomly stops responding and I have reboot. I'm running it through Virtual Box in a test environment.
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mar_becker on August 31, 2018, 10:58:11 am
Hello everybody,

I have a same problem.

suricata: [100165] <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"URLhaus Known malware download URL detected"; flow:established,from_client; content:"GET"; http_method; content:"/url=http://wordpress.p364918.webspaceconfig.de/614tiscfz/com/us|26|data=02|01|rcorm1@jcp.com|ec2a6ed25318490bd27608d6077bf11e|9c0ac0b90217468aa4322649cd6ed297|0|0|636704626242706015|26|sdata=g3qlynktc59ma3fllqbbfs0uwnigsem1mwi/cdfotvu=|26|reserved=0"; http_uri; depth:250; isdataat:!1,relative; content:"na01.safelinks.protection.outlook.com"; http_host; depth:37; isdataat:!1,relative; metadata:created_at 2018_08_21; reference:url, urlhaus.abuse.ch/url/45633/; classtype:trojan-activity;sid:80908733; rev:1;)^M" from file /usr/local/etc/suricata/opnsense.rules/abuse.ch.urlhaus.rules at line 3979

Is there already a solution?

Greetings

Mario
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: AdSchellevis on August 31, 2018, 11:50:17 am
Hi Mario,

Easiest fix is to temporary disable the broken rule, search for sid "80908733" in rules and disable. Then restart and you should be fine.

Best regards,

Ad
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mar_becker on August 31, 2018, 01:48:49 pm
Hi Ad,

Thanks for the hint. Now the error message does not appear anymore, but Suricata stops after a few minutes. In the log no further entry appears.Do you have any idea what this may be?

Thanks in advance.

Mario
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: jclendineng on August 31, 2018, 02:08:01 pm
I have the same issue, basically running suricata kills my opnsense box, forcing a hard reboot.  Im working on a suricata server I can pass all traffic through.  Does not look like it was meant to be.  Pfsense has this same issue, but the rules are much much easier to add/remove/suppress.  My solution is to spin up another server running a dedicated IDS firewall such as simplewall/SELKS/bare metal suricata (or even pfsense just running suricata), and pass the data to another server running ELK stack.  So Modem > Opnsense Firewall > IDS > LAN with a failover bypassing the IDS in case it goes down, and for devices I do not need to worry about using VLANS.  Was hoping to pass data directly to my log server but my elk stack is using docker which means no real way to input opnsense data. 
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: AdSchellevis on August 31, 2018, 02:42:31 pm
Hi Mario,

There seem to be other issues with the URLhaus ruleset, causing suricata to crash.
I can reproduce your issue, although I receive no useful errors either other then a coredump.

At the moment it's probably better to disable the URLHaus ruleset, I don't have the time at the moment to track down the real issue.

Best regards,

Ad
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mar_becker on August 31, 2018, 02:59:05 pm
Hi Ad,

thank you for your support. DonĀ“t worry....

Greetings

Mario
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: Werner Fischer on September 03, 2018, 02:37:25 pm
I can reproduce this issue, too. I'm also getting the error message:

Quote
Sep 3 13:59:36    suricata: [100118] <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"URLhaus Known malware download URL detected"; flow:established,from_client; content:"GET"; http_method; content:"/url=http://wordpress.p364918.webspaceconfig.de/614tiscfz/com/us|26|data=02|01|rcorm1@jcp.com|ec2a6ed25318490bd27608d6077bf11e|9c0ac0b90217468aa4322649cd6ed297|0|0|636704626242706015|26|sdata=g3qlynktc59ma3fllqbbfs0uwnigsem1mwi/cdfotvu=|26|reserved=0"; http_uri; depth:250; isdataat:!1,relative; content:"na01.safelinks.protection.outlook.com"; http_host; depth:37; isdataat:!1,relative; metadata:created_at 2018_08_21; reference:url, urlhaus.abuse.ch/url/45633/; classtype:trojan-activity;sid:80908733; rev:1;)^M" from file /usr/local/etc/suricata/opnsense.rules/abuse.ch.urlhaus.rules at line 5298

And in "clog /var/log/system.log":
Quote
Sep  3 14:01:45 OPNsense kernel: pid 6613 (suricata), uid 0: exited on signal 6 (core dumped)

Switching from Pattern matcher "Hyperscan" to "Aho-Corasick" resolves the issue that Suricata dies, but on a 100G WAN link the speed is dropping to 20G (small system with SSE3 capable Atom-CPU).

Disabling "abuse.ch/URLhaus" also fixes the issues, too (leaving "Hyperscan" activated).

I just wanted to provide those details, maybe this helps you to find the root cause of the issue (is not time critical to me).

Best regards,
Werner
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mar_becker on September 13, 2018, 10:03:18 am
Hi Werner

that was a good tip to disable the rules of abuse.ch. Since then Suricata runs without further problems.

Thanks and greetings

Mario
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mimugmail on September 13, 2018, 10:25:51 am
The author of the ruleset told me that he added a mechanism that rules are capped.
Perhaps it's now worth a try?
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mar_becker on September 14, 2018, 08:58:42 am
Hi everybody,
it seems to work. I have activated the rulesets of abuse.ch again and have not yet discovered any errors.

Greetings

Mario
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: JasMan on September 21, 2018, 04:33:07 pm
Hey,

I can't confirm that's working again. I've still the same error.

Jas Man
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: mimugmail on September 21, 2018, 04:39:04 pm
Did you updated them before enabling?
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: JasMan on September 21, 2018, 05:08:08 pm
Yes, I think so.
When I re-enable the ruleset and click "Download & Update rules", the "Last update" column shows me the actual date and time for abuse.ch/URLhaus. But right after that IPS chrashes and all other rulesets are not updated.

Is there another way to update the ruleset without enabling it?
Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: samsonmcnulty on November 21, 2018, 06:06:05 am
Edit:I've just seen https://forum.opnsense.org/index.php?topic=10268.0 (https://forum.opnsense.org/index.php?topic=10268.0) where the issue is addressed. apologies for the necro.

I wanted to bump this and say I'm still encountering the issue with the same rule set on version 18.7.7. Everything works just fine as long as that specific rule set isn't loaded. Re-downloading doesn't fix the issue.

Title: Re: IDS/IPS Surciata fails to start when abuse.ch/URLhaus is activated
Post by: bmail on November 21, 2018, 09:10:27 am
Hello,

Not the same issue. I was talking about downloading rules from feodotracker.abuse.ch or sslbl.abuse.ch.
Issue with URLhaus seems to be related to hypersan method.