ISP bandwidth is cut in 1/3 due to Suricata Intrusion detection IPS Mode

[sonic1812]

I have an opnsense running the following:
Board N100
Ram 32 GB
SSD:528
Version   25.7.11_2   
Architecture   amd64

With IPS mode OFF on suricata, I get 1300 mbps on speedtest
With IPS mode ON, I get ~500 mbps on speedtest.
I have all the hardware filtering settings on Interface turn OFF. See attached.

How can I get my provisioned bandwidth of 1300 mbps with IPS Mode ON?

[meyergru]

IPS mode is a lot more taxing than just routing and firewalling. And RSS mode is not applicable, because IPS is inherently single threaded, see the note here: https://docs.opnsense.org/troubleshooting/performance.html

The hardware settings will do next to nothing, if they work at all. Some seem to work first, with some subtle problems coming up later.

So, essentially, the answer is: Use a CPU with more single-thread punch.

[Greg_E]

Suricata should be multithreaded, it definitely was when I was running it on pfsense, and I'm guessing it is on OPNsense. Snort was single threaded for a long time, I think they may have fixed this by now (but not sure).

[franco]

> https://docs.opnsense.org/troubleshooting/performance.html#note-regarding-ips

I don't think this has been true for a while.  Asked Stephan to update the docs.


Cheers,
Franco

[meyergru]

If that is true, RSS mode should fix it.

[Seimus]

From what I have seen on forum Suricata should support Multicore.
This part of docs is indeed bit out of date :)

Regards,
S.

[Greg_E]

On pf long ago, Suricata was multithreaded which gave a performance boost over Snort.

As far as performance impact, if every rule is turned on, every rule must be checked and that takes time and RAM. Pass it through Zenarmor too and down it does.

With both Suricata and Zenarmor on my old low power Xeon based system (4c8t) and 16gb of ram, my gigabit connection give me about 600mbps down and we still get nearly gigabit up. More cores, faster clock, plenty of RAM seems to be the way to go. With modern i3 or n305 processors, you should out perform my firewall by a lot.

[Seimus]

More cores, faster clock, plenty of RAM seems to be the way to go.

Not if you use ZA, which is singlecore bound.

Regards,
S.

[franco]

> On pf long ago, Suricata was multithreaded which gave a performance boost over Snort.

The key part here is Netmap+Suricata.


Cheers,
Franco

[Seimus]

I will just add,

The improvements on Netmap done particularly the emulated netmap driver is just... another level its same or even better than the native one.

Regards,
S.

[glevai]

I have the same problem. The firewall worked well in january with full performance (almost 1 gigabit), but after an update (I don't know which, because I didn't check the performance until march) it halved the performance at 100% cpu usage. Suricata processes eat up the cpu (/usr/local/bin/suricata -D --netmap --pidfile /var/run/suricata.pid -c /usr/local/etc/suricata/suricata.yaml{W#03-igc1^}). If I remove all the rules (and apply, restart), the cpu load remains the same when I run speedtest.