21.7.5 IDS error

[Julien]

Hi Guys,

i've been waiting to upate the box to 21.7.5 as i was worried the IDS would crashes. unfortunately i have to upate the box so the Suricada has been updated too.
after the upate Suricada has ping point plenty of errors on the log.

are the below error something i have to worry about ?

Code: [Select]

2021-11-20T22:35:59 suricata[26424] [100374] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2021-11-20T22:35:59 suricata[26424] [101239] <Notice> -- opened netmap:em0/T from em0: 0x3e791c92300
2021-11-20T22:35:59 suricata[26424] [101239] <Notice> -- opened netmap:em0^ from em0^: 0x3e791c92000
2021-11-20T22:35:59 suricata[26424] [101230] <Notice> -- opened netmap:em0^ from em0^: 0x3e790a6d300
2021-11-20T22:35:59 suricata[26424] [101230] <Notice> -- opened netmap:em0/R from em0: 0x3e790a6d000
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.vba-jpg-dl' is checked but not set. Checked in 2814992 and 0 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.HTA.Download' is checked but not set. Checked in 2816701 and 0 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.autoit.ua' is checked but not set. Checked in 2019165 and 0 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ETPRO.certutilhttp' is checked but not set. Checked in 2833774 and 0 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.IE7.NoRef.NoCookie' is checked but not set. Checked in 2023671 and 2 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.http.binary' is checked but not set. Checked in 2023741 and 1 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.gadu.loggedin' is checked but not set. Checked in 2807836 and 0 other sigs
2021-11-20T22:35:41 suricata[26424] [100374] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.gocd.auth' is checked but not set. Checked in 2034333 and 0 other sigs
2021-11-20T22:35:37 suricata[16325] [100121] <Notice> -- This is Suricata version 6.0.3 RELEASE running in SYSTEM mode
2021-11-20T22:35:36 suricata[3667] [100451] <Notice> -- Stats for 'em0^': pkts: 5, drop: 0 (0.00%), invalid chksum: 0
2021-11-20T22:35:36 suricata[3667] [100451] <Notice> -- Stats for 'em0': pkts: 0, drop: 0 (nan%), invalid chksum: 0
2021-11-20T22:35:36 suricata[3667] [100451] <Notice> -- Signal Received. Stopping engine.
2021-11-20T22:35:36 suricata[3667] [100451] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
Also i notice there is a new button has been added to the Intrusion detection "Policy"
is the new way how to configure it those days? are those different from the Administration / Rules  ?

Thank you

[Fright]

Hi

the below error something i have to worry about ?

no. but that rules will just not triggered

flowbit 'ETPRO.certutilhttp' is checked but not set

before some flowbit can be matched it should be set. but the rule that sets this flowbit is not enabled (or does not exists)

s the new way how to configure it those days? are those different from the Administration / Rules  ?

yep. a new way to manage rules based on their metadata (allows to manage a large number of rules without overflowing the config file)

[Julien]

Thank you for your answer

yep. a new way to manage rules based on their metadata (allows to manage a large number of rules without overflowing the config file)

so the old tradition "Administration/Rules"  have to disable those there and keep using the policy ?
i also noticed there hasnt been any generated alerts.

[Fright]

have to disable those there and keep using the policy ?

I would say this is the desired method. but policies have limitations (if the rule does not contain suitable metadata, then there is no way to form a policy). so imho it remains to combine the use of policies where possible and management at the rules level where policies do not fit

[Julien]

have to disable those there and keep using the policy ?

I would say this is the desired method. but policies have limitations (if the rule does not contain suitable metadata, then there is no way to form a policy). so imho it remains to combine the use of policies where possible and management at the rules level where policies do not fit

Thank you for your answer.
i've noticed when i disable the rules i notice there is no meta data in the policy.
also i noticed the IDS stops working and i have to enable it manually. i cannot seems to find the cause.
the below is the log i can find

Code: [Select]

2021-11-22T11:29:56 suricata[43809] [100156] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2021-11-22T11:29:56 suricata[43809] [101348] <Notice> -- opened netmap:em0/T from em0: 0xf6685fd300
2021-11-22T11:29:56 suricata[43809] [101348] <Notice> -- opened netmap:em0^ from em0^: 0xf6685fd000
2021-11-22T11:29:55 suricata[43809] [101340] <Notice> -- opened netmap:em0^ from em0^: 0xf65344b300
2021-11-22T11:29:55 suricata[43809] [101340] <Notice> -- opened netmap:em0/R from em0: 0xf65344b000
2021-11-22T11:29:55 suricata[42445] [100216] <Notice> -- This is Suricata version 6.0.3 RELEASE running in SYSTEM mode
2021-11-21T23:02:21 suricata[5127] [1:2024930:1] ET WEB_SERVER 401TRG Generic Webshell Request - POST with wget in body [Classification: Web Application Attack] [Priority: 1] {TCP} 62.182.71.111:3003 -> 192.168.4.7:443
2021-11-21T21:00:11 suricata[5127] [100165] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2021-11-21T21:00:11 suricata[5127] [101086] <Notice> -- opened netmap:em0/T from em0: 0x247159fc300
2021-11-21T21:00:11 suricata[5127] [101086] <Notice> -- opened netmap:em0^ from em0^: 0x247159fc000
2021-11-21T21:00:11 suricata[5127] [101077] <Notice> -- opened netmap:em0^ from em0^: 0x24700310300
2021-11-21T21:00:10 suricata[5127] [101077] <Notice> -- opened netmap:em0/R from em0: 0x24700310000
2021-11-21T21:00:10 suricata[43706] [100355] <Notice> -- This is Suricata version 6.0.3 RELEASE running in SYSTEM mode
2021-11-21T21:00:09 suricata[86424] [100218] <Notice> -- Stats for 'em0^': pkts: 28183, drop: 0 (0.00%), invalid chksum: 0
2021-11-21T21:00:09 suricata[86424] [100218] <Notice> -- Stats for 'em0': pkts: 23682, drop: 0 (0.00%), invalid chksum: 0
2021-11-21T21:00:09 suricata[86424] [100218] <Notice> -- Signal Received. Stopping engine.
2021-11-21T20:59:23 suricata[86424] [100218] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2021-11-21T20:59:23 suricata[86424] [100969] <Notice> -- opened netmap:em0/T from em0: 0x1ec06761300
2021-11-21T20:59:23 suricata[86424] [100969] <Notice> -- opened netmap:em0^ from em0^: 0x1ec06761000
2021-11-21T20:59:22 suricata[86424] [100956] <Notice> -- opened netmap:em0^ from em0^: 0x1ebdc1fd300

[Fright]

imho you need to remove this rules from Services: Intrusion Detection: Policy->Rule adjustments in order to remove them from the __manual__ policy. after that it will be possible to manage them using new policies

[Julien]

Thank you for your answer.
Are you reffering to those two rules?
i havent added them, they shows up after the udpdate

[Fright]

hi

Are you reffering to those two rules?

yes. rules in this tab is out of policies (manual management)

i notice there is no meta data in the policy

hm. when you add new policy there should be different criterias for policy match
https://docs.opnsense.org/manual/ips.html#policies

if there is no choice of metadata, then something may have gone wrong

[Julien]

Thank you for your answers.
i am facing this issue that IDS stops working just like this i cannot seems to find any errors in the log.
when i enable it it start working again.
is there a reason why? or somewhere in the log ?

Thank you

[Fright]

hi

when i enable it it start working again.

sorry, may be i missing something.
after what specific steps did it stop working and after what steps did it start working again?

[Julien]

just casually, its just stops and when monit sent a email i log in to turn it on.
for now i've disabled both IDS and Dot which is crashes after the new new update.

i get pain in my stomach when those updates keeps showing up and quickly without testing them.

[Fright]

may be any clue in logs? high cpu or memory consumption?
(enabled suricata and unbound with DoT and all blacklists enabled on test VM. no issues yet)

[neo72]

Hi!
I have the same error. About one time a day the network communication stops working. Then i have to restart the suricata service and then it works again. i found nothing in the logs which could explain this problem. The machine is a Xeon e5-2620 with 32 GB Ram and 600 GB HDD, so there should be no problems. I use dual-WAN and IPS on WAN side and Sensei on LAN-side. So maybe it is a problem in the last release?
Greetings
Rudolf

[autone]

Hi!
I have the same error. About one time a day the network communication stops working. Then i have to restart the suricata service and then it works again. i found nothing in the logs which could explain this problem. The machine is a Xeon e5-2620 with 32 GB Ram and 600 GB HDD, so there should be no problems. I use dual-WAN and IPS on WAN side and Sensei on LAN-side. So maybe it is a problem in the last release?
Greetings
Rudolf

I have the same problem but no errors in the logs. Can't even SSH in and GUI unresponsive. Needs a reboot for everything to come back up.

Disabled suricata IPS and it's ok in IDS mode. Only enabling IPS mode causes this issue.

[Julien]

i have no log or what so ever to check this.
its just crashes and i have to enable it manually.


from time to time i see those on the log

Code: [Select]

2021-12-14T00:01:49 suricata[28934] [100135] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2021-12-14T00:01:47 suricata[28934] [100135] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] - signature at /usr/local/etc/suricata/opnsense.rules/emerging-worm.rules:48 uses unknown classtype: "command-and-control", using default priority 3. This message won't be shown again for this classtype
2021-12-14T00:01:47 suricata[59787] [100397] <Notice> -- This is Suricata version 6.0.4 RELEASE running in SYSTEM mode
2021-12-14T00:01:46 suricata[53460] [100420] <Notice> -- Signal Received. Stopping engine.
2021-12-14T00:01:37 suricata[53460] [100420] <Notice> -- rule reload complete
2021-12-14T00:01:33 suricata[53460] [100420] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] - signature at /usr/local/etc/suricata/opnsense.rules/emerging-worm.rules:48 uses unknown classtype: "command-and-control", using default priority 3. This message won't be shown again for this classtype
2021-12-14T00:01:33 suricata[53460] [100420] <Notice> -- rule reload starting