OPNsense Forum
Archive => 23.1 Legacy Series => Topic started by: Dexter_23 on March 27, 2023, 10:50:53 am
-
Hi
I need help to install and configure collectd for monitor cpu,ping etcetc.
For now i only install collectd on OPNSense but i don't know how to enable graphs like screenshoot i attach
-
Collect sends the data to some external time series database like InfluxDB or Graphite.
-
Hi
Did you see the screenshoot i have attached? Collectd Graph Panel
https://github.com/pommi/CGP
Is possible to install CGP on OPNSense directly?
-
It may be possible but the plugin is intended to be an agent only, not a central server
-
ok so the question is, what system did you use for monitoring collectd metrics?
-
InfluxDB and Grafana.
-
Ok did you have a tutorial for install and configure?
-
What platform? I use a FreeBSD jail.
-
I don't know what platform did you recommended to use? What is FreeBSD Jail?
-
You need a separate server. What operating system does that run? A VM would be fine. A jail is a FreeBSD native container that can be run on e.g. TrueNAS, which is what I do.
-
I have a Proxmox Server on Hetzner witch have a lxc container on docker engine on top or it's better to create a VM with Debian or Ubuntu Server?
-
Whatever you can use to run Influx and Grafana ...
To the influxd.conf you need to create a "graphite plaintext" compatible connector:
[[graphite]]
enabled = true
database = "opnsense"
retention-policy = ""
bind-address = ":2003"
protocol = "tcp"
consistency-level = "one"
separator = "."
templates = [
"servers.* .hostname.resource.instance.measurement*",
]
Then point your collectd at the influxd IP address, Port 2003. See screenshot for details.
Then point Grafana at Influx as a data source ...
-
Hi
Thank you for the advice, i think the easy and fast way is to deploy a container with docker for influxdb and grafana
-
@pmhausen
I deploy grafana e influxdb, i add influxdb database source to grafana, the influxdb service is listening on 8086/tcp, i don't understand why i need to edit influxdb.conf and add graphite listen on 2003/tcp, so i need to expose 2023/tcp port in addition 8086/tcp on influxdb container?
[[graphite]] >>>>> and this is an error 4 square brackets?
enabled = true
database = "opnsense"
retention-policy = ""
bind-address = ":2003"
protocol = "tcp"
consistency-level = "one"
separator = "."
templates = [
"servers.* .hostname.resource.instance.measurement*",
]
-
Collectd does not speak the native Influx protocol but both collectd and Influxd can speak the Graphite plain text protocol. So that's what we use. You can use a Graphite database instead of Influx but that is much more complicated to set up.
In Influx you need only the connector on port 2003 to receive data. I have to look up how Grafana communicates with Influx to retrieve data ...
The 4 brackets are correct at least for Influx 1.8.x - I don't have a 2.x installation available.
-
Ok so in Grafana what datasource i need to choose??(see screenshoot)
-
Influx on port 8086.
-
Hi i have add the database (see screenshoots), but this server has to be reach external, in portainer as you can see the port exposed to the host is only the 8086, i need to redeploy the container influxdb choose port 2003/tcp instead 8086/tcp?
-
I run Influx and Grafana here in my home lab on a dedicated server and they are thus reachable via a private IP address. I don't know what portainer is.
You will need both ports. Collectd sends to Influx on port 2003. Grafana reads from Influx on port 8086. If Influx and Grafana are running on the same system that can of course be 127.0.0.1:8086.
-
Hi
I've mapping both ports on the host running docker port tcp/8086 and tcp/2003 on influxdb container, on my opnsense firewall in collectd i have configure like the screenshoot you send me, i download a dashboard for opnsense in grafana dashboard website and import the dashboard but i don't receive data from OPNSense Firewall.
-
UPDATE:
I see one problem after edit the infludb container for mapping port 2003/tcp now the configuration file in /etc/influxdb/influxdb.conf lost the changes for graphite connector.
How to make persistant if i need to edit the container?
-
Sorry, I have no idea. This is container specific and has got nothing to do with Influx or even OPNsense related questions. I run the software on a dedicated server and when I edit configuration files they stay that way.
You should probably ask in some container specific community.
I don't want to be rude, but this is definitely out of scope here.
EDIT: P.S. If you want a one-stop solution to just get some graphs, may I recommend Observium? https://www.observium.org
-
Off course is not related to OPNSense, so don't worry :)
This is the actual file /etc/influxdb/influxdb.conf:
[meta]
dir = "/var/lib/influxdb/meta"
[data]
dir = "/var/lib/influxdb/data"
engine = "tsm1"
wal-dir = "/var/lib/influxdb/wal"
[[graphite]]
enabled = true
database = "opnsense"
retention-policy = ""
bind-address = ":2003"
protocol = "tcp"
consistency-level = "one"
separator = "."
templates = [
"servers.* .hostname.resource.instance.measurement*",
]
I attach screenshoot about OPNSense CollectD Configuration and Grafana Dashboard, as you can see i don't receive data from collectd.
-
Can you use shell commands inside that thing that runs Influx? If yes, please try:
influx
use opnsense;
show series;
Ctrl-D
-
Hi
The command you tell me to use working correcty (see the screenshoot)
-
Then it's an issue with the data source or the dashboard definition. How did you create the dashboard?
-
Hi
I import a dashboard for OPNSense by searching in "https://grafana.com/grafana/dashboards/"
see the screenshoot for details
Anyway if you have a nice dashboard for opnsense can you send me the code for import :)
Thanks
-
The dashboard configuration depends heavily on the data source and in the case of InfluxDB on the template that is used to dissect the data fed to it into different labels.
If the dashboard you used uses labels different from
- hostname
- resource
- instance
then you will not see anything. These labels are something I configured specifically for my needs graphing TrueNAS performance data. It will work with OPNsense, too, since it's both "just FreeBSD", but the Grafana dashboard will have to match.
There is no "click here" standard to get graphs. All of this is in the responsibility and control of the user. You can come up with any naming scheme you like.
I'll try to come up with something for OPNsense over the weekend.
Kind regards,
Patrick
-
Hi
I don't know how to configure dashborad to receive data(see screenshoots)
-
HI @pmhausen
Any good news?
I can't understand how to configure dashboard to receive data...
-
Haven't found the time yet. You need to learn Grafana and InfluxQL if you want to create dashboards.
If you want to use a complete dashboard you downloaded from some source you need to ask the author which time series database (Influx, Graphite, ...) they are using and how their template is set up.
Grafana is not a one stop solution.
-
Ok but in my case what type of dashboard i need for my scenario?
InfluxDB o Graphite?
-
Influx since you are using Influx as the time series database.
But then this depends on the template. Dashboards are in no way "standardised". What exactly do you want to achieve in the end?
Collectd just sends a series of entries to the time series database. They are structured like this:
servers.foo.bar.baz.quux <measurement> <timestamp>
The template in the influxdb config says that each line starts with the literal "servers" and that the following fields are named "hostname", "resource" and "instance" respectively.
You can then reference these fiels in Grafana like in my screenshot.
"Select the field value of gauge.system" from the default data source (InfluxDB) where the hostname matches the host selected by the user, the instance is named "cpu-average" and the resource is named "aggregation". You get these things from the "show series" output in Influx. That shows you all that is available.
Again, there is no one stop dashboard. If you name the field "foo" instead of "instance", you must use "foo" in Grafana - it's all sort of like "programming".
You probably might want to start some reading on Grafana. There's a Grafana forum, too.
-
i don't have thies field like your screenshoot i think...