OPNsense Forum

English Forums => Intrusion Detection and Prevention => Topic started by: geotek on April 19, 2025, 01:54:24 PM

Title: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: geotek on April 19, 2025, 01:54:24 PM
We are using the proofpoint rules and all OPNsense versions from 25.1 to 24.x are affected. Error message is:

<Error> -- Just ran out of space in the queue. Fatal Error. Exiting. Please file a bug report on this

It looks like a broken rule update is responsible for this, since ample memory and disk space is available on our boxes.
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: geotek on April 20, 2025, 11:25:21 AM
One more observation: Only boxes with one of the three Aho-Corasick Pattern matchers are affected, even with today's updated rules. Boxes with Hyperscan matcher were not affected. After changing the matcher to Hyperscan, the problem was solved on all of our previously affected firewalls.

I hope this helps identifying and fixing the cause.
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: allenlook on April 21, 2025, 02:47:02 PM
Ours started erroring out on the 20th, same symptoms, and same temporary resolution - Hyperscan.
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: ErikLievense on April 22, 2025, 01:58:34 AM
Same here
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: Greg_E on April 22, 2025, 03:25:18 PM
Mine is set to "default" and still working. This reminds that I need to sit and do some work on my filters and same for Crowdsec.
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: bogardon on April 27, 2025, 12:31:25 AM
Mine is set to default and it's still crashing. Any updates on a fix?
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: VG Gerolstein on April 30, 2025, 01:49:53 PM
Same here...
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: Ametite on May 03, 2025, 06:49:25 PM
Same message here, latest version installed
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: mokaz on May 05, 2025, 08:28:29 AM
Had the same issue, here is what fixed it for me:


According to the docs, Hpyerscan seems to be the best options whenever supported, I'll leave it at that here.
https://docs.opnsense.org/manual/ips.html

Kind regards,
m.
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: someone on May 06, 2025, 07:28:15 PM
Same here and its still broke
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: someone on May 06, 2025, 11:42:56 PM
I changed IPS>Administratiom>Settings Advanced and changed pattern matcher to Hyperscan
As pointed out by user geotek
And Detect profile to medium, may not have needed to change that
Its working for now
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: carlos on May 07, 2025, 11:39:52 AM
I promise this question is honest. I don't want to make enemies... please do not be too agressive with the answer..

I have moved from pfSense to OPNSense 2 months ago and now I'm facing this issue.

I activated proofpoint, it was great.

Now, suricata stops work and there is no solution several weeks later.

My question is ... Could this be a prove that OPNSense is more modern, with more functionalities but it's not being maintained as fast pfSense?
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: peter.vynck on May 10, 2025, 03:08:00 PM
Hi, this solution works for me! Thanks.



Quote from: someone on May 06, 2025, 11:42:56 PMI changed IPS>Administratiom>Settings Advanced and changed pattern matcher to Hyperscan
As pointed out by user geotek
And Detect profile to medium, may not have needed to change that
Its working for now
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: joeyboon on May 10, 2025, 09:52:29 PM
I also wanted to report the same issue and the fact that this solution worked for me.

Quote from: geotek on April 20, 2025, 11:25:21 AMOne more observation: Only boxes with one of the three Aho-Corasick Pattern matchers are affected, even with today's updated rules. Boxes with Hyperscan matcher were not affected. After changing the matcher to Hyperscan, the problem was solved on all of our previously affected firewalls.

I hope this helps identifying and fixing the cause.

Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: relume on May 30, 2025, 08:38:58 PM
Hello

Started today with IDS on OPNSense 25.1.7_4 and selected "Hyperscan" as pattern matcher. Unfortunately, I got with "Hyperscan" the error "IDS log reports "hs" is an invalid mpm algo"  and it became apparent that "Hyperscan" requires SSSE3 and actually running OPNSense on Proxmox with qemu64 on (old) Xeon Westmere EP hardware, it is not possible to switch the cpu type to any else that qemu64 to start the OPNsense VM. Switching to "Aho-Corasick, Ken Steele" resulted then in the above error suricata "Error - Just ran out of space in the queue. Fatal Error."
Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: agromsl on June 04, 2025, 05:47:55 AM
Mokaz's solution worked.

As a preface, I must say I could never activate IPS with hyperscan matcher on this VM.
That VM presented some other limitations compared to its twin, almost identical, except for the CPU type.

After reading Mokaz's solution I did not go for same CPU as him but simply abandoned the KVM64, for the «Broadwell, IBRS», the one used on the almost twin VM and it solved the problem.
(and I'll keep an eye on any difference that may appear or persist between the twin VMs)

Thank You!!!

Clearly, KVM64 CPU emulation misses flags required by hyperscan to perform.

An embryo of CPU compatibility list includes:


Quote from: mokaz on May 05, 2025, 08:28:29 AMHad the same issue, here is what fixed it for me:

  • updated the vCPU scheme of the VM from "kvm64" to "Haswell-noTSX".
  • VM power off/power on.
  • shifted the IPS engine from "Aho–Corasick Ken Steele variant" to "Hyperscan" (only possible post point #1 here).

According to the docs, Hpyerscan seems to be the best options whenever supported, I'll leave it at that here.
https://docs.opnsense.org/manual/ips.html

Title: Re: Suricata is crashing on all firerwalls starting April 19th, 2025
Post by: agromsl on June 04, 2025, 02:44:52 PM
... and a wiser way to describe compatibility would be to enumerate required CPU flags, instead of listing all variants of CPU released on the market.