Suricata IPS 10Gbps

[dcol]

Had a few minutes to get this together.
Here is a sample iperf3 going from the firewall to Windows PC on the physical LAN. 192.168.100.2 is the PC and 192.168.100.1 is the firewall. Let me know if I should be using other endpoints.
LAN is using x710-DA4 2-10GB ports setup as lacp lagg0. WAN is Intel i210.
CPU is i5-7600, RAM 16GB

[seed]

So you have Suricata running on your gigabit interface. But you claim that you reach 10 gigabit throughput. Your screenshot even proves that your statement is not correct (2,8Gb). Also, the requirement was to route the traffic through the OPNsense. Sorry but you missed the point.

[mimugmail]

Also ... you are using IPerf from LAN interface to a LAN host while Suricata only runs on WAN. :)

[dcol]

First off, I never said I achieved 10GB speeds. I just stated that it works. If I had better instructions on what you wanted to see maybe you would have what you wanted. My goal was to start a conversation about how to improve IDS performance not a condemnation. I just wasted my time with this thread. Thanks.

[dcol]

Here are some comparisons, using IDS on LAN only and 10GB NIC's on both LANs
Even without IDS, I can only achieve around 6 Gb/s, so IDS doesn't slow it down too much.
IDS is using 4 rulesets. Same computer specs and NIC's on both sides.

[Supermule]

Remember that your SATA bus doesnt push more than 6gbit/s no matter what.

So many of the systems sold cannot push more than that.

SAS pushes 12gbit/s and Nvme is limitless. (more depending on NIC's and CPU).

[dcol]

Using NVMe not SATA on both systems

[seed]

Remember that your SATA bus doesnt push more than 6gbit/s no matter what.

So many of the systems sold cannot push more than that.

SAS pushes 12gbit/s and Nvme is limitless. (more depending on NIC's and CPU).

This thread is getting spammed by people who completely miss the topic.
Can the moderators close this topic?

It may take some cpu generations until 10gbps IPS are in reach. Until then this discussion goes nowhere.

[lilsense]

to answer your own question, get a threadripper with 10Gig card and see if you can make it sweat. :D

[Supermule]

Remember that your SATA bus doesnt push more than 6gbit/s no matter what.

So many of the systems sold cannot push more than that.

SAS pushes 12gbit/s and Nvme is limitless. (more depending on NIC's and CPU).

This thread is getting spammed by people who completely miss the topic.
Can the moderators close this topic?

It may take some cpu generations until 10gbps IPS are in reach. Until then this discussion goes nowhere.

So because you dont agree or dont like, then you ask for a closure....

It can easily be done. Servergrade hardware (Dual Xeon's) and I710-T4 nics. This is what we use. It just keeps tugging along at about 1,4MM PPS hardly breaking a sweat.

[Patrick M. Hausen]

What does disk bandwidth - though factually correct - have to do with IPS performance?

[Supermule]

What does disk bandwidth - though factually correct - have to do with IPS performance?

Primarily log writing to disk.... we used this as a guide.

https://redpiranha.net/news/High-speed-IDP/S-suricata-hardware-tuning-for-60gpbs-throughput

https://www.google.dk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjP_YvxsKb8AhUdRvEDHRqBCVIQFnoECD8QAQ&url=https%3A%2F%2Fuia.brage.unit.no%2Fuia-xmlui%2Fbitstream%2Fhandle%2F11250%2F2823637%2FF%25C3%25B8rde%2520Roar%2520%2528705%2529_78839715_2.pdf%3Fsequence%3D1&usg=AOvVaw2YPOejlIrJWikYDIc32L6E

[Patrick M. Hausen]

But with 10 Gbps network to scan as the OP asked, and 9X% of all traffic being irrelevant - do you really think SATA could ever become a bottleneck?

You don't log unsuspicious/permitted connections, do you?

[Supermule]

But with 10 Gbps network to scan as the OP asked, and 9X% of all traffic being irrelevant - do you really think SATA could ever become a bottleneck?

You don't log unsuspicious/permitted connections, do you?

It becomes a bottleneck when Suricata writes to the logs no matter the ruleset/traffic.

In "the other sense" as soon as it sees above the 200.000 PPS mark it becomes sluggish because of the disk subsystem and the logging...

[ryanhaver]

I've looked into this a lot...and admittedly, it's hard to find up-to-date and reliable information. From everything I have investigated, it is even more challenging to get close to 10Gbps IPS using Suricata on FreeBSD because of Netmap.

Although Sucircata can utilize more than one CPU core, Netmap's implementation on FreeBSD has historically been limited to a single CPU core when using Suricata in IPS mode. Apparently, there is work underway to change this behavior, but I haven't been able to find the current state of progress.

This was previously brought up by a forum admin in the post I've quoted below. It has been almost two years since the post though...so I'm on the hunt for any updates on this.

Hi,

Suricata on FreeBSD uses Netmap to achieve IPS functionality. Judging by your logs, you are indeed using netmap to bypass the host stack and enable Suricata to inspect packets straight off the wire.

Note the way ports are opened:

ix0/R (Receive thread) --> ix0^ (Host stack)
ix0^ (Host stack) --> ix0/T (Transmit thread)

This simply means that on initialization, netmap opens two "ports" - one on which to capture packets, at which point Suricata will be able to do it's thing, and another port that represents the host stack (using the '^' symbol), which is used by Suricata to forward inspected packets back to the host stack. The same principle applies on the transmit side (but reversed) - totalling a thread usage of 4 in a default setup.

The way Netmap is currently implemented does not allow for more than one thread to connect to the host stack on both the receive and transmit side. Manually increasing the amount of threads will not ensure a gain in throughput, and any measured increase in throughput will be wrong, since packets on different threads might not even reach Suricata and thus could potentially even skip by Suricata, due to a lack of synchronization.

In conclusion, Suricata on FreeBSD currently only supports one thread in IPS mode. However, Netmap has recently committed support for multiple threads towards the host stack in FreeBSD, and Suricata is in the process of integrating this into their software - so keep an eye on that.

Cheers,

Stephan