IDS/IPS drop of internet speed

[xmichielx]

Hello,

I am very happy with the OPNsense 17.1.6 amd64 box I am running on a PCengines APU2d4 (4 cores, 4 GB memory and 3 gigabit interfaces with a microSSD of 14 GB)
I am trying out the IDS/IPS to block malware using the ET malware/trojan/shellcode and 3 more plugins + the 4 SSL (gedotracker etc.) plugins and the OPNsense test plugin.
All works well and the eicar.txt is happily dropped when the IPS is enabled and default action is drop.
Whenever I try to download a large file, for example an ISO from ftp.nluug.nl, the speed is capped at 5 MB/s wheres my max speed would be around 17,3 MB/s.
I have tracked it down to the IDS/IPS/Suricata service and disabling the service gives full blown internet again but I was wondering if the IDS/IPS could really gave such a performance hit knowing that there is not a lot of traffic going through (it's an ziggo cable connection at home so not much users or devices), the APU2D4 has 4 GB ram and 4 cores (suricata can use all cores if I'm not mistaken) and I just enabled a few plugins.
Perhaps also good to know is that I have a VLAN1 interface linked to the igb0 (or LAN interface) which is 1 GB + have enabled netflow + insight.

Is this normal behaviour and should I really run a very expensive i7 hectacore with 32 GB of RAM or should the APU2D4 be able to track its traffic?

Cheers and thanks

Michiel

[netranger]

Hi

I have the same box as you and same version of OPNSense.
I experienced similar behaviour. The following ruleset used a big chunk of my performance:
ET open/emerging-trojan

Did not test this one but i read that this is quite performance-hungry as well:
abuse.ch/Dyre SSL IPBL

If I just use a few specific ones I don't see a big performance impact.

Cheers

[xmichielx]

Cool!
I will try that, did you also changed the default to Hyperscan? I see some improvements there too.

[xmichielx]

I did not find an improve, perhaps it did but very small..still <10 MB/s with almost everything disabled except the malware ones + the OPNsense test plugin..

[netranger]

Hmm no I never played with that, always used "aho-corasick". Have you tried with this?

You could disable one after another until you find the one which is the problem.
I have 9 active ruleset with almost full bandwidth.

Cheers,

[xmichielx]

I tried all 3 options and also disabling plugins and enabling them one by one, I do see an increase in bandwidth after changing suricata options like syslog or promiscuous settings but then it drops to 8-10 MB/s.
I also tried disabling Netflow to save some resources but to no effect.
I also experienced a strange thing with the OPNSense eicar rule: It blocks the eicar.txt download the first time, when I refresh the website (hard refresh) and download the eicar.txt file again it allows it to download.
I really have to refresh my cache of my browser to let it block again..this is imho unwanted behaviour as users could retry their download and succeed in downloading potential malware.

[Noctur]

Enable RAM disk? System - Settings - Miscellaneous - /var RAM Disk, /tmp RAM Disk

Running a quad i5 getting >100mb/s throughput with a huge number of Suricata rules and Geo-IP. 8gb Ram, 120gb SSD.

I also have firewall rules with Geo-IP alias defined and blocked, preventing them from ever getting to Suricata, reducing the stream volume to be scanned. Also have Hyperscan selected as recommended earlier.

[xmichielx]

Is there a way I can troubleshoot this since all mentioned options sound/feel like a shot in the dark..
top shows 130% CPU but since I have 4 CPU's I don't expect that should be an issue?
Memory wise I have 2 GB free and the microssd should also be fast enough..

[xmichielx]

I also tried to change the workers to autofp but that decreased the speed even more -> https://forum.opnsense.org/index.php?topic=4683.msg20289#msg20289

I also noticed that no alerts are triggered when I run multiple portscans against my monitored device (WAN), same behaviour : https://forum.opnsense.org/index.php?topic=4937.msg19895#msg19895

So I got an IPS that detects the OPNsense default ruleset but no others are triggered and a decrease in internet speed which I can not explain

[csmall]

I have the same issue but have never been able to figure why. I just switched to an Intel server quad port nic on new hardware and same issue.

*shrugs*

[netranger]

Someone else with real knowledge of this module has to help you here. All I can say is I am not sure if a port scan should trigger an alarm anyway, at least I don't have a rule for it. But I have other rules alerting all the time, for example File Tracking GIF when surfing.

[xmichielx]

I did a reinstall with Pfsense and found out that:

a) it's normal that your bandwidth is being capped - same results with Suricata on OPNSense and PFsense with IPS an Netmap/Hyperscan enabled
b) PFsense alerts are shown under Alerts, not with OPNsense *except* for the OPNSense test rules

A nmap is being triggered by ET scan rules on PFSense, not on OPNsense.

I like OPNSense more but if no alerts are triggered with the same box + setup (suricata setup on FreeBSD with the same suricata settings via the GUI - not sure what is done in the config files) then I rather choose PFsense then OPNsense for the sake of stopping bad traffic and when needed, drop traffic related to ransomware.

My 0,02$.

[xmichielx]

Another finding: Snort with default settings and the same plugins on Pfsense does no cap my bandwidth (aka 120/150 Mb/s).
I am now using snort and understood that multi threaded is very interesting for 1GB+ bandwidth.
Suricata and IPS is capping too much bandwidth from my available bandwidth.

[franco]

Snort does not have an IPS mode, just a loosely coupled IP block mechanism to the firewall. It's the same as Suricata for IDS mode, except for the blocking because we do not believe it is secure enough, because in data loss prevention this is completely ineffective as it blocks too late.

Remember, you are trading security for speed. Choose according to your needs or push the hardware platform if the secure option is slower than required.


Cheers,
Franco

[xmichielx]

Hi Franco,

Thanks for the reply.
Why is it too late? Have you tested the difference in speeds of the block between Suricata inline and Snort?
So you're saying that the IPS functionality with Snort is *always* too late - the session is already created and data already posted before Snort and the IP block mechanism could do its work? (which I find interesting since some companies offer Snort as IPS for companies).
Also the IPS on OPNsense did not show any alerts except for the the eicar download testcase..
So I regained my speed, got some alerts (at least) which is better then drop in speed (50%) and and no alerts.

I have the APU2b4 and those have Intel nics (https://www.pcengines.ch/apu2c4.htm -> Intel i211AT on apu2b2, i210AT on apu2b4 which would be the same on the apu2c4) and I see that they are supported by netmap but still I see such a decrease in bandwidth..
Shouldn't netmap cover the resource issue and shouldn't the 4 GB memory + 4 cores of the APU2C4 be enough to do inline processing?