1
21.7 Legacy Series / Intrusion Detection Alerts not sent by Telegraf
« on: July 30, 2021, 03:23:00 pm »
* OPNsense 21.7-amd64
* os-telegraf 1.11.0 (Telegraf 1.19.0)
* InfluxDB 2.0.7
I was hoping this release would fix the problem I have with Intrusion Detection Alerts not beiing sent to InfluxDB.
https://opnsense.org/opnsense-21-7-released/ states that "intrusion detection: fix alert reads from eve.json", but as I was unable to find any more information in the release post, this statement is probably referering to another issue. Anyways, to bump this issue to 21.7, I'll post my findings here on this more active Forum section - I have rambled over at the General Discussion page quite a bit : https://forum.opnsense.org/index.php?topic=16966.0)
1) Suricata is started with the user root
2) Suricata produces events in the /var/log/suricata/ directory which has these permissions
A quick fix is to add the user telegraf to the wheel group and change permissions. This will not survive a reboot.
Then metrics are populated in InfluxDBv2.
Although I have not figured this out entirely, I think the [[inputs.tail]] section would benefit from having "timestamp" from eve.json parsed.
https://docs.influxdata.com/telegraf/v1.19/data_formats/input/json/#json_time_key-json_time_format:
Has anyone got this to work / having the same problems?
* os-telegraf 1.11.0 (Telegraf 1.19.0)
* InfluxDB 2.0.7
I was hoping this release would fix the problem I have with Intrusion Detection Alerts not beiing sent to InfluxDB.
https://opnsense.org/opnsense-21-7-released/ states that "intrusion detection: fix alert reads from eve.json", but as I was unable to find any more information in the release post, this statement is probably referering to another issue. Anyways, to bump this issue to 21.7, I'll post my findings here on this more active Forum section - I have rambled over at the General Discussion page quite a bit : https://forum.opnsense.org/index.php?topic=16966.0)
1) Suricata is started with the user root
2) Suricata produces events in the /var/log/suricata/ directory which has these permissions
Code: [Select]
drwx------ 2 root wheel 12B Jul 30 00:00 suricata
3) Suricata creates JSON entries in the file /var/log/suricata/eve.json which has these permissionsCode: [Select]
-rwx------ 1 root wheel 30K Jul 30 14:53 /var/log/suricata/eve.json
4) Enabling "Intrusion Detection Alerts" in Telegraf, creates this config in /usr/local/etc/telegraf.confCode: [Select]
[[inputs.tail]]
data_format = "json"
files = ["/var/log/suricata/eve.json"]
name_override = "suricata"
tag_keys = ["event_type","src_ip","src_port","dest_ip","dest_port"]
json_string_fields = ["*"]
5) Telegraf is started by the user telegrafCode: [Select]
ps aux | grep telegraf
telegraf 12093 0.0 1.1 5040852 92304 - S 14:02 0:35.47 /usr/local/bin/telegraf --quiet --config=/usr/local/etc/telegraf.conf --config-directory=/usr/local/etc/telegraf.d
6) Telegraf does not have permissions to view the file /var/log/suricata/eve.jsonCode: [Select]
sudo -u telegraf more /var/log/suricata/eve.json
/var/log/suricata/eve.json: Permission denied
7) There is no errors in /var/log/telegraf/telegraf.log from [[inputs.tail]] that the file is inaccessibleA quick fix is to add the user telegraf to the wheel group and change permissions. This will not survive a reboot.
Code: [Select]
pw group mod wheel -m telegraf
chmod 750 /var/log/suricata ; chmod 750 /var/log/suricata/eve.json
Then metrics are populated in InfluxDBv2.
Although I have not figured this out entirely, I think the [[inputs.tail]] section would benefit from having "timestamp" from eve.json parsed.
https://docs.influxdata.com/telegraf/v1.19/data_formats/input/json/#json_time_key-json_time_format:
Quote
By default the current time will be used for all created metrics, to set the time using the JSON document you can use the json_time_key and json_time_format options together to set the time to a value in the parsed document.
The json_time_key option specifies the key containing the time value and json_time_format must be set to unix, unix_ms, or the Go “reference time” which is defined to be the specific time: Mon Jan 2 15:04:05 MST 2006.
Code: [Select]
[[inputs.tail]]
data_format = "json"
files = ["/var/log/suricata/eve.json"]
name_override = "suricata-alerts"
tag_keys = ["flow_id","in_iface","event_type","src_ip","src_port","dest_ip","dest_port","proto"]
json_string_fields = ["*"]
json_time_key = "timestamp"
json_time_format = "2006-01-02T15:04:05-0700"
Has anyone got this to work / having the same problems?