OPNsense Forum
Archive => 15.7 Legacy Series => Topic started by: Solaris17 on August 16, 2015, 01:58:40 am
-
Hi! I have recently installed OPNsense and currently over the 20 hours it has been running I have 11GB of log files. My SSD is only 32GB when I go into settings the feild for log file size is blank. I dont mind it using the space (not much else to use it for really) but will this start to cause issues as the drive gets mroe full? will opnsense have enough space provisioned for updates via another partitions? will old logs be deleted to make room for new logs?
do I have a problem I should correct?
-
Did you set it to log everything?
-
Did you set it to log everything?
I don't believe so. I also cleared the logs and the space actually didn't go down at all. So it seems log files arent even the culprit. at 21 hours in currently im at 13gb used from 11 when I made the OP.
Took a closer look via SSH
using df -h in shell I was able to confim the usage.
I then used du -h to see what directory was using the space. turns out its suricata. I have everything checked (selected) and updated. There seems to be no setting to control log size. would anyone know how to clear them? maybe I have a configuration issue?
tried to rm /var/log/suricata/status.log which removed it however the space is the same. killed syslog (cant reboot atm) i'll reboot the machine when the network isnt so busy. Still have the space condition. apparently suricata logs every 8 seconds by default.
rebooting the unit had no effect :(
tracked it down. ran du -k on everything in the suricata dir and found out it was actually eve.json taking up the space. I'm looking into what if anything can be done about tis size.
I just deleted it for now. Looks like suricata is supposed to have a log limit size of 32mb but then rolls over and starts a new one. it appears to rely on anotehr program to clean up old log files.
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml
Suricata in itself however does have a configuration that im unsure how to getto or modify (BSD makes me cry)
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml
a really neat feature though that would be cool if implemented though is support for logtash
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/_logstash_kibana_and_suricata_json_output
its a nifty gui that might be useful in parsing the actual suricata logs.
Im still trying to understand how we can limit the size though.
-
You probably have quite some alerts if the file is growing that fast, maybe it's a good idea to look if all alerts produced are really that valuable for your situation.
We use the eve format for logging, because it is very good usable for integration. Our gui uses it for some basic search and analyse features, but (installed on a different machine) the logstash/kibana combination should deliver more features.
A gui on logstash is probably not something we're going to provide, but it might be an idea to add logstash to our pkg repository at some point (you could add a feature request for that on github in the tools directory).
Log rotates are handled by newsyslog, which keeps it's configuration in /etc/newsyslog.conf.d/suricata.
At this file is generated by a (at the moment rather static) template in /usr/local/opnsense/service/templates/OPNsense/IDS/newsyslog.conf
The defaults are to rotate once a day and keep 7 days of data.
Unfortunately we don't deliver a configuration option for this newsyslog template (yet), so an upgrade puts back our standards at the moment.
(but you could experiment with other settings if you like).
When you apply the settings in the gui, the templates are automatically generated and your new defaults should be used then.
-
Thanks! I will comb through the options too see exactly what I dont need and this should curb the log usage. Its a shame I deleted it honestly. I would be interested to know which one was filling the log that quickly.