Topics

1

24.7 Production Series / WARNING: failed to start ntopng

« on: August 27, 2024, 12:58:48 am »

Code: [Select]

2024-08-27T00:48:43 Warning ntopng 27/Aug/2024 00:48:43 [Ntop.cpp:3890] WARNING: Unable to find timezone: using UTC
2024-08-27T00:48:40 Notice root /usr/local/etc/rc.d/ntopng: WARNING: failed to start ntopng
2024-08-27T00:48:40 Warning ntopng 27/Aug/2024 00:48:40 [Ntop.cpp:3890] WARNING: Unable to find timezone: using UTC
2024-08-27T00:47:27 Notice root /usr/local/etc/rc.d/ntopng: WARNING: failed to start ntopng
2024-08-27T00:47:27 Error ntopng 27/Aug/2024 00:47:27 [Redis.cpp:157] ERROR: to specify a redis server other than the default
2024-08-27T00:47:27 Error ntopng 27/Aug/2024 00:47:27 [Redis.cpp:154] ERROR: Please start it and try again or use -r
2024-08-27T00:47:27 Error ntopng 27/Aug/2024 00:47:27 [Redis.cpp:153] ERROR: ntopng requires redis server to be up and running
2024-08-27T00:47:26 Error ntopng 27/Aug/2024 00:47:26 [Redis.cpp:98] ERROR: Connection error [Connection refused]
2024-08-27T00:47:25 Error ntopng 27/Aug/2024 00:47:25 [Redis.cpp:98] ERROR: Connection error [Connection refused]
2024-08-27T00:47:24 Error ntopng 27/Aug/2024 00:47:24 [Redis.cpp:98] ERROR: Connection error [Connection refused]

After latest update. ..
Tried stopping and starting redis, tried to stop and start ntopng, tried to reset Redis as well. Tried a reboot.

And ideas?

2

21.7 Legacy Series / Intrusion Detection Alerts not sent by Telegraf

« on: July 30, 2021, 03:23:00 pm »

* OPNsense 21.7-amd64
* os-telegraf 1.11.0 (Telegraf 1.19.0)
* InfluxDB 2.0.7

I was hoping this release would fix the problem I have with Intrusion Detection Alerts not beiing sent to InfluxDB.

https://opnsense.org/opnsense-21-7-released/ states that "intrusion detection: fix alert reads from eve.json", but as I was unable to find any more information in the release post, this statement is probably referering to another issue. Anyways, to bump this issue to 21.7, I'll post my findings here on this more active Forum section - I have rambled over at the General Discussion page quite a bit : https://forum.opnsense.org/index.php?topic=16966.0)

1) Suricata is started with the user root
2) Suricata produces events in the /var/log/suricata/ directory which has these permissions

Code: [Select]

drwx------   2 root      wheel       12B Jul 30 00:00 suricata
3) Suricata creates JSON entries in the file /var/log/suricata/eve.json which has these permissions

Code: [Select]

-rwx------  1 root  wheel    30K Jul 30 14:53 /var/log/suricata/eve.json
4) Enabling "Intrusion Detection Alerts" in Telegraf, creates this config in /usr/local/etc/telegraf.conf

Code: [Select]

[[inputs.tail]]
  data_format = "json"
  files = ["/var/log/suricata/eve.json"]
  name_override = "suricata"
  tag_keys = ["event_type","src_ip","src_port","dest_ip","dest_port"]
  json_string_fields = ["*"]
5) Telegraf is started by the user telegraf

Code: [Select]

ps aux | grep telegraf
telegraf 12093   0.0  1.1  5040852   92304  -  S    14:02     0:35.47 /usr/local/bin/telegraf --quiet --config=/usr/local/etc/telegraf.conf --config-directory=/usr/local/etc/telegraf.d
6) Telegraf does not have permissions to view the file /var/log/suricata/eve.json

Code: [Select]

sudo -u telegraf more /var/log/suricata/eve.json
/var/log/suricata/eve.json: Permission denied
7) There is no errors in /var/log/telegraf/telegraf.log from [[inputs.tail]] that the file is inaccessible

A quick fix is to add the user telegraf to the wheel group and change permissions. This will not survive a reboot.

Code: [Select]

pw group mod wheel -m telegraf
chmod 750 /var/log/suricata ; chmod 750 /var/log/suricata/eve.json

Then metrics are populated in InfluxDBv2.

Although I have not figured this out entirely, I think the [[inputs.tail]] section would benefit from having "timestamp" from eve.json parsed.

https://docs.influxdata.com/telegraf/v1.19/data_formats/input/json/#json_time_key-json_time_format:

By default the current time will be used for all created metrics, to set the time using the JSON document you can use the json_time_key and json_time_format options together to set the time to a value in the parsed document.

The json_time_key option specifies the key containing the time value and json_time_format must be set to unix, unix_ms, or the Go “reference time” which is defined to be the specific time: Mon Jan 2 15:04:05 MST 2006.

Code: [Select]

[[inputs.tail]]
  data_format = "json"
  files = ["/var/log/suricata/eve.json"]
  name_override = "suricata-alerts"
  tag_keys = ["flow_id","in_iface","event_type","src_ip","src_port","dest_ip","dest_port","proto"]
  json_string_fields = ["*"]
  json_time_key = "timestamp"
  json_time_format = "2006-01-02T15:04:05-0700"

Has anyone got this to work / having the same problems?

3

21.1 Legacy Series / [SOLVED] Problems with Telegraf and InfluxDB v2

« on: July 20, 2021, 02:37:11 pm »

• OPNsense 21.1.8_1-amd64
• os-telegraf (installed)   1.11.0
• Telegraf 1.19.0   
• VLAN_15: 192.168.15.0/24

• Ubuntu 20.04.2 LTS
• InfluxDB 2.0.7 (git: 2a45f0c037) build_date: 2021-06-04T19:17:40Z

I am not certain how this is supposed to work, but I do not receive any data from OPNsense to my InfluxDB (??). There is nothing showing on the Boards section of InfluxDB, "No Results", using the default "System" dashboard and the correct bucket. Using the "Explore" section of InfluxDB, there is no data- even when enabling "View Raw Data".  I do not know what I have done wrong. Can someone help me?

Output

• Enable Influx v2 Output: checked
• Influx v2 URL: http://192.168.15.10:8086/api/v2/telegrafs/07da8386caf81000
• Influx v2 Token: II4dVNinSKLxllAPQ2AoU18Gf3XY23sTmKM0rAXsH8FqFmHT3YQ-jyxPnslaT7rq-4HMW-mUVpmmug5WpS0bvQ==
• Influx v2 Organization: help
• Influx v2 Bucket: test

The Input section is default. I have then on the General tab enabled the Telegraf Agent. I have saved all these settings, and I've also tried to restart the Telegraf service. Both the Quiet and Debug Log have been enabled.

Using

Code: [Select]

netstat on the host machine (192.168.15.10), I can see a tcp connection from the firewall on port:8086, but no data has been sent/received. The installation of InfluxDB is done following this guide: https://docs.influxdata.com/influxdb/v2.0/install/?t=Linux on an Ubuntu 20.04 machine.

I have double, triple and quadrupled checked that the bucket is correct, the Token is correct and the Organization name is correct (which has been altered in this forum post).

The most common error I've found in the OPNsense Telegraf Log File is

Code: [Select]

2021-07-20T12:22:50 E! [outputs.influxdb_v2] Failed to write metric (will be dropped: 404 Not Found): not found: path not found

However, when I change a setting / hit Save in the Input section of Telegraf, I get these errors:

Code: [Select]

2021-07-20T12:29:00 I! Starting Telegraf 1.19.0
  time="2021-07-20T14:29:00+02:00" level=error msg="failed to open. Ignored. open /.cache/snowflake/ocsp_response_cache.json: no such file or directory\n" func="gosnowflake.(*defaultLogger).Errorf" file="log.go:120"
  time="2021-07-20T14:29:00+02:00" level=error msg="failed to create cache directory. /.cache/snowflake, err: mkdir /.cache: permission denied. ignored\n" func="gosnowflake.(*defaultLogger).Errorf" file="log.go:120"

Are those errors related to why nothing is happening on my InfluxDB v2 instance? I have tried to remove the plugin, install the plugin and even rebooted the firewall.

The firewall (192.168.15.1) is able to ping the machine hosting InfluxDB, from the VLAN15 interface (Source Address) - there are no funky firewall rules either. ..

What have I done wrong?