Using Rulesets in Suricata IPS

[dcol]

I would like to start a new topic based on how to get the most from IPS rulesets.

Let me explain why we care about ruleset selections when using IPS.
IPS uses netmap which is a method of capturing packets using circular queues of buffers (netmap rings) implemented in shared memory. In short, netmap can inspect packets before they are delivered to the OS. This 'inspection' is where rulesets are used. The list of rulesets are created from the rulesets you picked and are formulated by a pattern matching algorithm (ie, Hyperscan) into a signature engine. Suricata loads signatures with which the network traffic will be compared using netmap to control packets before they are delivered to the firewall. The size and efficiency of this 'engine' determines how much processing Suricata needs to do. By design, as soon as a signature is 'matched' the inspection ends. So imagine all the good packets have to go through the entire engine before it is released. Suricata loads signatures from this engine using netmap to control packets before they are delivered to the firewall.

We want netmap to do as little as possible because of the resources required to do this work. The firewall can do the grunt work. But keep in mind that quality is better than quantity when it comes to rules.

Choosing IPS rulesets is based on your needs. An email server behind OPNsense does not require the same rulesets as desktop internet users behind OPNsense. Choose wisely as the efficiency will depend on it. Do not use rulesets that do not apply to your usage. I refrain from listing any recommended rulesets since this opens up too much controversy.

Here are links to ruleset explanations. You decide which ones you should enable.
ET Rules: http://tools.emergingthreats.net/docs/ETPro%20Rule%20Categories.pdf
Snort Rules: https://www.snort.org/rules_explanation

Also take this info into consideration as recommended by Suricata, not me.

These rulesets are useful but often high load rules. Look here for performance tuning
- emerging-web_client.rules
- emerging-netbios.rules

Rules you'll want to look through and consider based on needs
- emerging-policy.rules # this ruleset can create a lot of false positives
- emerging-games.rules
- emerging-p2p.rules
- emerging-chat.rules

Informational rulesets, not recommended for high speed nets
- emerging-icmp_info.rules
- emerging-info.rules
- emerging-shellcode.rules # very noisy
- emerging-inappropriate.rules
- emerging-web_specific_apps.rules
- emerging-activex.rules

Once you have all your rules enabled, you need to edit each ruleset and select 'Change all alerts to drop action'. It is also recommended to monitor the IPS alerts for a while, especially during peak usage times, to see if any legitimate traffic is being blocked. Also check the Suricata log to insure that there are no signature errors. Disable any rules that shows an error. Snort Rulesets are primarily designed for Snort and will produce some errors with Suricata.

###################################################
USEFUL SHELL COMMANDS
kill -USR2 <PID> # Silently reload the Suricata rules. Get the Suricata <PID> from System>Activity
###################################################

If you want to add your own custom ruleset to OPNsense then follow this tutorial
https://forum.opnsense.org/index.php?topic=7209.msg32271#msg32271

[elektroinside]

[comment not valid anymore]

Respectfully, I think this list is/was constructed with anything but security in mind. I cannot see the signature of a security analyst here just be looking at the CVEs many of the "not recommended / should stay away from" rules cover.

While I care for performance and throughput quite a lot, over the 10+ years working with security products (as in actively involved in the development of such products), where vulnerabilities, exploits, viruses and all sorts and forms of malicious activities were involved, including clients (people - humans - many mistakes) that our products protected (which naturally meant research about what is worth developing - selling points but also quality features), i can conclude:

1. There cannot be a list of vulnerabilities one can just ignore
2. There are (and will be) measures to prevent attacks that will interact with legitimate data and overall performance, one may need to manually supervise
3. There is no such thing as a security guru to deploy good security; there is, however, a level of security one may be aware of (or not), proved to be sufficient (until it's not)
4. Back to the list: many rules here are old, will protect against old vulnerabilities most of you probably already patched, but there is absolutely nobody in this world (well, almost) who will tell you to ignore them without any serious research on the matter, in your (you, the end-user) particular case, for your particular environment, without an assessment; it's just not healthy

Again, the point of my comment is to raise your awareness about security best practices, which is, among others, to analyze the information you get from here and there. I do not wish to get into a debate with the author (or anybody) nor to disrespect his work, as I'm sure this list needed some googling. At the end of the day, it's not my digital environment that these rules (or the absence of them) will secure (or not).

But.. please don't play with your security. You will never know when and how it will let you down.

I do apologize to dcol if me or my comment wronged him in any way.

Cheers!

[dcol]

I would be glad to see your recommendations. I am just trying to get a much needed conversation started.
My recommendations are a starting point for someone using OPNsense. Not meant for security professionals.
And most definitely not a complete solution or end all. But I think you missed my point. IPS is just a front end to weed out stragglers. The firewall can do the dirty work. If you bog down IPS it will greatly impede performance and possibly bring down OPNsense. IPS should not substitute or replace a firewall with solid rules. At least OPNsense has IPS. Most other solutions out there only have IDS or broken IPS because they are keeping Snort compatibility..

One other point I am trying to make is that one needs to be cautious on choosing rules to use. If one were to just install all the rules, then one may have issues like the ones I have seen posted.

[elektroinside]

[comment not valid anymore]

As I said, I prefer not to debate, nor discuss reasons, because of one simple thing: you can't. The term 'security' is relative (in this context), and because of this, I can only recommend anybody to use a product to its full extent.

The solution will never be to disable half of the functionality because it's not working properly. If it isn't, find the fault and try to fix it somehow, or in the worst case, try to work around the issue. You see, I wrote issue, not issues (plural), it was intentional. Take it step by step, I did not cut the entire product in half. If you have to cut the product in half for it to work, quit, find another job, or you can always simply choose to just fix the damn issue. This is how things work in the software business (from a technical pov at least).

It's not the security specialist's fault, who has spent many hours to discover the vulnerability, get to build an accurately functioning PoC and in the end write the rule, that my windows updates service is not working properly. It's mine, because either I cannot use it properly, or it simply isn't compatible with WU and I did not choose which one to keep. It's my choice whether or not to disable the rule or continue using it as it is. I will never recommend disabling random rulesets (because they are random) to anybody.

If OPNsense is falling apart by the use of something they choose to integrate, they should fix OPNsense, not dismiss half of the other product. If they can't, they should not integrate it. You just can't have it both ways. It's either working or it's not. If limits need to be applied, that's a different story and a long development road to decide how many things and what to limit. But there aren't any limits. Just to be clear, I fiddled a lot with IDS in the OPNsense implementation lately and OPNsense never failed on me. Suricata might have, but never OPNsense.

The firewall is one thing, the IDS is another. They are entirely two different products. They both do the same thing, drop stuff, but on entirely different levels. It's just like comparing an airplane with a bicycle. Both carry passengers, but in completely different ways. In other words, you can't replace a bicycle with an airplane. I definitely did not say that. The firewall can't do the things an IDS does, it's not the same logic behind the two. So although I understand what you wanted to say, I can't comprehend the logic behind it, as you can't technically compare the two.

I'm not here to compete, demonstrate anything, play mind or word games, measure proudness levels and I'm yet to surprise my mother-in-law. I've grown out of such things.

[mimugmail]

Hey guys .. first of all, I'm not the regular IPS user but IMHO it's as always "it depends".

I'd only use IPS with some kind of paid rulles to do virtual patching against 0days as everyone with mission critical data should keep systems up2date.

@dcol: I'm not sure if you have to copy all rules in rules and opnsense.rules. Normally rules folder is just fine and depending on enabled/disabled to file get's copied/removed to/from opnsense.rules

Ah, I think I already told you I did some kind of low level L7 detection to gain minimal feature parity like OpenAppID. Go to /usr/local/etc/suricata/rules and:

fetch https://raw.githubusercontent.com/opnsense/rules/master/src/opnsense.file_transfer.rules
fetch https://raw.githubusercontent.com/opnsense/rules/master/src/opnsense.social_media.rules
fetch https://raw.githubusercontent.com/opnsense/rules/master/src/opnsense.messaging.rules

Reload IPS and then you have some more categories in the rules tab. E.g. dropdown to messaging and you get rules to block messaging apps. Beware, these are only http/dns/https-sni rules, so there's a chance to still use the apps, but for normal users good to block unwanted applications. We'll add some more categories in Q2, you can follow this on:

https://github.com/opnsense/core/issues/1887

[dcol]

It really comes down to your needs. For me I don't need to block messaging or social media. My firewalls are usually just protecting email and web servers, not users.

I am happy with country blocking and my golden rules posted above.

But my real point is, unless you have a powerhouse computer with gobs of memory, your firewall will come crashing down if you apply too many rules to IPS. Thats why I posted a selection of must have rules to keep it simple for users who want to get started using IPS. You may not agree with them but its a good start.

[elektroinside]

[comment not valid anymore]

My memory consumption is (and it is not that relevant as I have many services activated):

Mem: 164M Active, 375M Inact, 594M Wired, 48M Buf, 6517M Free
70294 root             8  20    0  1720M   293M nanslp  3   4:19   1.22% suricata

LAN clients online right now: 14 (among them 2 servers)
Why is this so bad?

I don't agree with the idea of cherry-picking rules under the pretty strong title of "recommended". I don't see a "by who", I don't see references, studies, nothing. I do agree, however, of explaining to users the possibility of high CPU/Mem usage if this and that. And I also agree with building a list of rules which might compromise/brake critical/common services/operations, without recommending to disable them. Let the user choose. Your post, as it is, is dangerous, especially to newbies.

Personally, I strongly advise users to consider the following:
1. IDS/IPS requires a small learning curve (including hardware requirements)
2. The best and fastest way to learn is to embrace the problems which might arise (connectivity mostly), all of them, one by one, and try to find the cause and then fix them (by disabling rules they don't need)
3. I do not recommend disabling entire rulesets, but rather individual rules, the ones that negatively impacts your environment (because there will be, depends on your infrastructure how many)
4. Once your IPS rules are settled, managing IPS will be easy and on a less frequent basis.

In this way, you will no longer be responsible (morally at least) of weakening the security of your readers.

[elektroinside]

@mimugmail:
Awesome work!

But this isn't working for me, I get this for each of the 3 new rulesets:

Code: [Select]

21/1/2018 -- 08:18:37 - <Error> - [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] - Unknown Classtype: "social-media".  Invalidating the Signature


[mimugmail]

@mimugmail:
Awesome work!

But this isn't working for me, I get this for each of the 3 new rulesets:

Code: [Select]

21/1/2018 -- 08:18:37 - <Error> - [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] - Unknown Classtype: "social-media".  Invalidating the Signature


The cool thing is you (or the community) just have to update the .lst files which are really easy to read.
Then a little script converts it to the ruleset.

Classification get's static values dynamically:
https://github.com/opnsense/core/blob/master/src/opnsense/service/templates/OPNsense/IDS/classification.config

We'll rework this.

To not get overwritten you have to add the values in:
/usr/local/opnsense/service/templates/OPNsense/IDS/classification.config

[dcol]

I have rewritten my original post and eliminated any personal ruleset recommendations based on the responses. I thought this would have helped beginners, not flare up security professionals.
All the info I supplied now is well documented from Suricata sources.

[elektroinside]

If it means anything, I think you did the right thing. A properly written piece of documentation (which we can extend, if you want to).

Thank you for taking into consideration my advices.

[networkguy]

I have been running in IDS mode for a while and I am about to switch to IPS. Where do blocked ip addresses or flows show up once I enable IPS. If I find a false positive how do I remove the block?

Thanks.

[elektroinside]

Services: Intrusion Detection: Administration: Alerts tab (click on the pencil to edit the action)

[networkguy]

If you change an alert from deny to alert does that just affect that flow or does it change the rule itself?

[elektroinside]

It affects the rule itself.
Don't forget to apply (go to the Rules tab and click Apply -> otherwise it will not use the new action).