OPNsense Forum
Archive => 19.1 Legacy Series => Topic started by: GDixon on December 01, 2018, 07:46:51 am
-
Hi,
I followed the wiki to enable Intrusion detection and have a couple problems.
OPNsense 19.1.b_306-amd64
FreeBSD 11.2-RELEASE-p4-HBSD
OpenSSL 1.0.2q 20 Nov 2018
I get these errors and of the 4 abuse.ch rule sets only the one actually downloads. These are the only rules I enabled to test suricata out.
abuse.ch/Dyre SSL IPBL not installed drop
abuse.ch/Feodo Tracker 2018/12/01 1:31 drop
abuse.ch/SSL Fingerprint Blacklist not installed drop
abuse.ch/SSL IP Blacklist not installed drop
and these errors are in the log
Dec 1 01:30:43 suricata: [100244] <Notice> -- Stats for 'em1': pkts: 283, drop: 0 (0.00%), invalid chksum: 0
ec 1 01:30:23 suricata: [100172] <Error> -- [ERRCODE: SC_ERR_INVALID_YAML_CONF_ENTRY(139)] - Invalid mpm algo supplied in the yaml conf file: "hs"
I rebooted and the same happens along with the IPv6 gateway being down and needing a restart.
what did i screw up this time? :)
-
Change between Hyperscan and Aho?
-
I'm using the default Aho but did try the hyperscan and ips/ids wouldn't load or refresh at all with hyperscan so am currently using the default Aho.
-
All of the abuse.ch lists have problems at the moment (server-problems caused from an OS-update).Try the ET Open lists.
-
I was going through old threads and saw that, went to their site and no mention of continuing problems. I'll turn off the 4 and try your suggestion :)
which ET rules would you recommend?
thank you
-
I turned off the 3 and added 5 of the ET rules at random to test and they downloaded and the few checks i made and they seem to work.
I still get the dhcp6, gateway dhcp6 and ntp problem and have to restart them to get IPv6 back.
The rules work and I'll see in a bit if the errors went away :)
again thank you!
EDIT: I cleared the log so we will see what comes back.
I set up cron in the gui when you close the pop up to see if the rule is added it goes to the last place you were in the configs . I was looking at the alerts before I did the cron and cron just put me right back to alerts. the gui will not let you look to see what rules are in the schedule.
I went to download the eicar test and it let me know it was a virus so that works but I don't see any blocked in the alerts. I used this site ( http://www.wicar.org/test-malware.html ) and it also blocked the flash and java script by only opening completly blank pages. I use firefox and run opensuse tumbleweed.
I probably should also mention I'm only using the WAN and not the lan if it makes any difference?
-
I was going through old threads and saw that, went to their site and no mention of continuing problems. I'll turn off the 4 and try your suggestion :)
which ET rules would you recommend?
thank you
Good that the main problem is now solved :-).
I have a "test prodecure" to check if an ruleset is working:
I click on the ruleset info symbol and open the URL in the "Ruleset details" field in a browser.Most of the abuse.ch URLs bring an HTTP "Error 503 Connection timed out".But that works only for the abuse.ch rulesets, other rulesets pointing to informational pages explaining the rulesets.
I have all of the other rulesets enabled.
And be careful to enable "abuse.ch/URLhaus", when they work again, because this ruleset can crash OPNsense (my experience).
-
...
I went to download the eicar test and it let me know it was a virus so that works but I don't see any blocked in the alerts. I used this site ( http://www.wicar.org/test-malware.html (http://www.wicar.org/test-malware.html) ) and it also blocked the flash and java script by only opening completly blank pages. I use firefox and run opensuse tumbleweed.
I probably should also mention I'm only using the WAN and not the lan if it makes any difference?
I cannot test the wicar.org site because Microsoft Windows Defender let my open this :-).
But i know there is no so-called "blockpage" informing you about the incident, only a blank site with timeout, so its probably blocking.
Maybe here is some more info:
clog -f /var/log/suricata.log
tail -f /var/log/suricata/stats.log
ATM I dont have enabled the LAN interface in suricata, only DMZ and WAN.
But i see sometimes things are blocked.
Some of the rulesets with malicious actions i set generally to Drop, not only Alert, for example "ET open/emerging-malware" or " ET open/emerging-trojan".
-
I saw where the haus one is a system crasher lol
All of them? I only did 5 to see how it works and so far I have yet to have anything show up in alerts.
CPU usuage is still minimal, memory usage did go up and the load avg went up very slightly so I think I'll load a few at a time and go from that.
Ah ha, I have things set to drop. So you need something set to alert for it to show in the gui?
So the alerts tab is more for testing to see what rules are messing up and son't really need to be dropped?
I see the only choice is none and drop so I'm going to guess none logs them in alerts?
-
...
All of them? I only did 5 to see how it works and so far I have yet to have anything show up in alerts.
CPU usuage is still minimal, memory usage did go up and the load avg went up very slightly so I think I'll load a few at a time and go from that.
Yes atm I have all of them enabled (exception all abuse.ch rulesets). CPU load is 28-31% with that with my hardware.
Ah ha, I have things set to drop. So you need something set to alert for it to show in the gui?
So the alerts tab is more for testing to see what rules are messing up and son't really need to be dropped?
Thats a good question!
In the "download" tab you can only enable the complete rulesets, then this ruleset is alerting. In the ruleset itself you can set them only to "None" or "Drop" and "None" means its alerting.
BUT: In the ruleset itself some or somtimes many of the signatures are not enabled.
In the "Rule" tab you can fine-tune the signatures: Disable completly, enable Alert or set to Drop.
You can see this when you search in the "Rule" tab for "trojan" (when you have enabled "emerging-trojan.rules". Some are enabled, some not.
And i see "eicar" is not enabled or it has the class-type "bad-unknown".
I assume class-type means here the status of the IPS-signature and bad is not so good :-)
I disabled in the last weeks/months myself some of the the signatures because they alerting about harmless events to reduce the noise of suricata.
I think the best is to surf and work as always and look from time to time in the logs to get a feeling for the whole thing :-).
-
Yes I found the second place to either drop or alert the rules. There's over 16,000!
How in the heck do you enable them all ? There has to be a quick way to do so with out scrolling through them all I hope.
And I still didn't download/enable all the ET rules hmmmmm
-
Yes I found the second place to either drop or alert the rules. There's over 16,000!
How in the heck do you enable them all ? There has to be a quick way to do so with out scrolling through them all I hope.
And I still didn't download/enable all the ET rules hmmmmm
Oh, sorry for the misunderstooding, I neverever enable them all (to alert)! :-)
In fact I enabled some of them manually over the GUI as described above.
I mean to know (no proof) that the maintainer of these rulesets enable the signatures that are important (from their view).
-
Doh
-
Doh
Dont worry!
When you want to test it out, download kali linux in a virtual machine and run some enumeration tools against the firewall itself or another (test) target. Before testing enable suricata at the LAN interface, to detect this.
-
I have new errors lol different than before and no alerts show up yet.
I'm going to remove surica, restore a good config and start over with surica.
this is just a few of the errors, there are many more similar.
Dec 1 11:27:33 suricata[66339]: [100275] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.armwget' is checked but not set. Checked in 2024241 and 1 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'min.gethttp' is checked but not set. Checked in 2023711 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.autoit.ua' is checked but not set. Checked in 2019165 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.MCOFF' is checked but not set. Checked in 2022303 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ETPRO.RTF' is checked but not set. Checked in 2020700 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.MSSQL' is checked but not set. Checked in 2020569 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.DocVBAProject' is checked but not set. Checked in 2020170 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.ELFDownload' is checked but not set. Checked in 2019896 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'etpro.clsid.detected' is checked but not set. Checked in 2002172 and 0 other sigs
Dec 1 11:27:28 suricata[66339]: [100275] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.MS.WinHttpRequest.no.exe.request' is checked but not set. Checked in 2022653 and 0 other sigs
-
I set up cron in the gui when you close the pop up to see if the rule is added it goes to the last place you were in the configs . I was looking at the alerts before I did the cron and cron just put me right back to alerts. the gui will not let you look to see what rules are in the schedule.
AH ha moment. you may not be able to look cron up in the suricata admin page but i found where cron is in System settings. your just not re directed while in the suricata admin page
silly me :) that part is solved
I tried to do a restore to factory and reload my config and that went well except now the web pages load slow especially the dashboard hmmmm
Next is to do a fresh format and reinstall and reconfigure by hand and see if it's all the rules that slow down the pages.
more tomorrow.
the no alerts showing problem is tentatively solved see the below posts,
https://forum.opnsense.org/index.php?topic=10193.0
slowly we go, i'll get it all figured out some day.
-
I have new errors lol different than before and no alerts show up yet.
I'm going to remove surica, restore a good config and start over with surica.
Good idea, never seen this errors before.
-
reformatted, installed, updated and reconfigured by hand.
enabled all the ET rules and yes it is the rules that slow the gui page loads, especially any that have live data like the dashboard or any that make significant changes so it is NOT a OPNsense problem.
no errors currently but a few things like the cron redirect do not work, no alerts show in the alerts admin gui and on reboot with ips/ids enabled some services tale a long time to load / start like dhcp6, ra ( related to dhcp6 most likely) ntp and the gateways.
Some of this is obviously a hardware limit. I'm using a old acer with a AMD athlon core 2 at 2.5 Ghz (4850e) with a 250G WD blue 2.5" HDD sata and no hyper threading or aes-ni and 4 Gigs of ram. Load is ok, ram usage 24 to 36% and temps 38 to 49 so ok Mbuf's at 1 %. State table has not gone above 1% so far
There is no noticeable slow down in browsing web sites or on the Lan (2 nas's (nas4free/Xigma), 3 cell phones and 5 to 7 computers on the Lan) that I can tell it mostly has to do with the OPNsense gui being slow to load in some areas.
When the alerts are fixed I'll go through and fine tune them and see what the thresh hold for this hardware might be with number of rules and speed for the OPNsense gui loading and then move to another hardware set up in time.
Possibly a dell r210 II or maybe I'll try one of the HP T620 plus thin clients.
For now it's just waiting for the next updates :)
EDIT: the 7 to 8 second refresh on the dashboard scrolls the dashboard back to the top and that does get ummm inconvenient when your trying to watch a graph or something below.
EDIT: I have these plugins enabled
os-dyndns (installed) 1.10_1 134KiB Dynamic DNS Support
os-smart (installed) 1.5 15.2KiB SMART tools
os-upnp (installed) 1.2_3 31.2KiB Universal Plug and Play Service
os-vnstat (installed) 1.0 20.7KiB vnStat is a console-based network traffic monitor
os-wol (installed) 2.0 20.8KiB Wake on LAN Service
whats the difference for the devel plugins?
greg
-
This might be a silly question.
Should the Wan or Lan be used?
I ask because the daughter had a question that makes one think.
If your monitoring the Wan it would mean your mis configured and not blocking what needs to be blocked if you get alerts so wouldn't it be better to monitor the Lan?
Well I switched to the Lan and now were seeing alerts which means nothing has made it through the Wan to cause alerts.
Which is best policy? Monitor the Wan? Monitor the Lan? Or choose both Wan and Lan to monitor?
-
...
Which is best policy? Monitor the Wan? Monitor the Lan? Or choose both Wan and Lan to monitor?
Try it out! :-)
Enable both (LAN/WAN) and monitor it, then you see whats going on...
-
I've tried all 3 and currently monitoring both wan and lan.
lots of allows for Lan and very very few for wan so I would guess monitoring both works.
No slow downs detected so left it monitoring both wan and lan for now.
Only thing I noticed is when watching the traffic graph is output for in as usual but for out is a straight line at the middle of the graph (0) for IPSEC with dots for traffic.
Now to figure out what can be dropped thats not a false positive and that will take a lot of research.