OPNsense Forum

Archive => 18.7 Legacy Series => Topic started by: chenwanli on October 07, 2018, 11:11:06 AM

Title: telegraf feature request
Post by: chenwanli on October 07, 2018, 11:11:06 AM
Hi,
I set up a opnsense appliance box as my homelab equipment,I noticed that telegraf could ONLY monitor metrics such as system,RAM,NET etc....

I also set ntpd,unbound(DNS resolver), could you make telegraf more customize so I could monitor more metrics ?

Thank you !
Title: Re: telegraf feature request
Post by: mimugmail on October 07, 2018, 11:43:03 AM
How would this .conf look like?
Title: Re: telegraf feature request
Post by: chenwanli on October 07, 2018, 12:12:07 PM
via telegraf config GUI
Title: Re: telegraf feature request
Post by: bringha on October 07, 2018, 05:52:00 PM
Good point - If I may also come up with a wishlist:

[[inputs.conntrack]] (if feasible on freebsd)

[[inputs.swap]]

[[inputs.hddtemp]]

[[inputs.ipmi_sensor]]

[[inputs.netstat]]

[[inputs.nginx]]

[[inputs.pf]]

[[inputs.smart]]

[[inputs.sysstat]] (if supported by freebsd)

[[inputs.unbound]]

[[inputs.zfs]]

Some of those need indeed some extension of the freebsd system packages also for OPNsense, which today might not be there to keep it small; but for professional use this might be worth thinking ....

Br br
Title: Re: telegraf feature request
Post by: mimugmail on October 07, 2018, 05:53:48 PM
bringha, can you enable these in telegraf.conf and restart the service via CLI and look if new metrics are collected? Sometimes there are additional configurations needed, like socket path
Title: Re: telegraf feature request
Post by: bringha on October 07, 2018, 06:06:17 PM
Yea, at least partly - will do when I'm back
Title: Re: telegraf feature request
Post by: fabian on October 07, 2018, 06:50:49 PM
conntrack will probably not work because that is a component of netfilter (the Linux firewall framework on which nftables and iptables are based).
Title: Re: telegraf feature request
Post by: chenwanli on October 08, 2018, 09:38:04 AM
telegraf itself supports many metrics,and opnsense has it's packages also。

I wish OPNsense could support monitoring the metrics of supported packages,like ntpd, unbound....

many people set up OPNsense as a base service device, gateway as it , ntp server as it, DNS as it....so monitoring these metrics may be wonderful!
Title: Re: telegraf feature request
Post by: mimugmail on October 08, 2018, 10:18:33 AM
I know, I just asked if you have tested it with OPNsense doing this without GUI, so I dont have to test everything by myself before adding it to plugin
Title: Re: telegraf feature request
Post by: chenwanli on October 08, 2018, 11:59:39 AM
yes, I manually inputs.ntpq into telegraf.conf ,it works...
Title: Re: telegraf feature request
Post by: mimugmail on October 08, 2018, 12:30:42 PM
What about the other values?
Title: Re: telegraf feature request
Post by: chenwanli on October 08, 2018, 01:41:32 PM

[[inputs.ntpq]]


just add above into telegraf.conf ,then manully start telegraf,it works well.

maybe you could read the telegraf inputs doc for full help.

https://github.com/influxdata/telegraf#input-plugins


Title: Re: telegraf feature request
Post by: bringha on October 08, 2018, 09:29:18 PM
Hi,

here some more feedback. Expanding some more features might be worth to consider some security implications

[[inputs.ipmi_sensor]]
servers = ["<ADMIN_USER>:<password>@lan(192.168.1.X)"]

works basically if the user telegraph is made belonging to group 'operator', otherwise /dev/ipmi0 can not be opened. Indeed, full ipmi_tool installation including kernelmodules need to be there. Could be a security issue.

[[inputs.pf]]
requires access to /dev/pf and user telegraf need to belong to group 'proxy' too; also worth a security consideration

[[inputs.netstat]]
needs command lsof which is in /usr/ports but requires kernel sources to compile; perhaps worth to consider to make lsof integral part of the standard installation. Might be that additional topics pop up after lsof has been installed.


[[inputs.unbound]]
## If running as a restricted user you can prepend sudo for additional access:
#use_sudo = false

## The default location of the unbound-control binary can be overridden with:
binary = "/usr/local/sbin/unbound-control"

## The default timeout of 1s can be overriden with:
timeout = "1s"

## Use the builtin fielddrop/fieldpass telegraf filters in order to keep/remove specific fields
fieldpass = ["total_*", "num_*","time_up", "mem_*"]

This requires enablement of usage of /usr/local/sbin/unbound-control to work in the unbound config. Did not have the time to get this up as certificates for client and server need to work properly but should be feasible basically. (Was not couragous enough to run unbound-control-setup  and to put my running config at risk on my productive system ....) ;)
There are some comments in the fora recommending not to enable unbound-control on a primary firewall installation.

All telegraf functions relying on /proc (eg /proc/CPUinfo) are likely to fail as freebsd proc has a widely smaller structure compared to Linux

Br br
Title: Re: telegraf feature request
Post by: chenwanli on October 12, 2018, 12:27:31 PM
cool, hope opnsense could make it . ;D ;D
Title: Re: telegraf feature request
Post by: mimugmail on October 12, 2018, 01:17:00 PM
I'll start after 18.7.5 .. have to sort out some things first, but most of it should be easy.
THe more you gave me input like this and that works when manually edit telegraf.conf the faster I can fix/add this. :)
Title: Re: telegraf feature request
Post by: bringha on October 14, 2018, 01:45:13 PM
Some more:

also interesting would be

[[inputs.statsd]]

because it allows a bunch of statistics around tcp connections et al.

However, although being contained in the ports tree of opnsense as described here https://forum.opnsense.org/index.php?topic=2004 (https://forum.opnsense.org/index.php?topic=2004) (indeed upgraded to 18.7.) it does not compile as some dependencies can not be built

===>  Staging for statsd-0.7.2_1
===>   statsd-0.7.2_1 depends on package: node6>=0 - not found
/!\ WARNING /!\

Ports Collection support for your FreeBSD version has ended, and no ports are
guaranteed to build on this system. Please upgrade to a supported release.

===>  License MIT accepted by the user
===>   node6-6.14.4 depends on file: /usr/local/sbin/pkg - found
=> node-v6.14.4.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://nodejs.org/dist/v6.14.4/node-v6.14.4.tar.gz
node-v6.14.4.tar.gz                           100% of   25 MB 2690 kBps 00m10s
===> Fetching all distfiles required by node6-6.14.4 for building
===>  Extracting for node6-6.14.4
=> SHA256 Checksum OK for node-v6.14.4.tar.gz.
===>  Patching for node6-6.14.4
===>  Applying FreeBSD patches for node6-6.14.4
===>   node6-6.14.4 depends on executable: gmake - found
===>   node6-6.14.4 depends on file: /usr/local/bin/python2.7 - found
===>   node6-6.14.4 depends on package: pkgconf>=1.3.0_1 - found
===>   node6-6.14.4 depends on file: /usr/local/lib/libcrypto.so.9 - found
===>   node6-6.14.4 depends on shared library: libcares.so - found (/usr/local/lib/libcares.so)
===>   node6-6.14.4 depends on shared library: libuv.so - not found
===>   libuv-1.23.2 depends on package: autoconf>=2.69 - found
===>   libuv-1.23.2 depends on package: automake>=1.16.1 - found
===>   libuv-1.23.2 depends on executable: libtoolize - not found
/!\ WARNING /!\

Ports Collection support for your FreeBSD version has ended, and no ports are
guaranteed to build on this system. Please upgrade to a supported release.

===>  License GPLv2 accepted by the user
===>   libtool-2.4.6 depends on file: /usr/local/sbin/pkg - found
=> libtool-2.4.6.tar.xz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://ftpmirror.gnu.org/libtool/libtool-2.4.6.tar.xz
libtool-2.4.6.tar.xz                          100% of  950 kB 2461 kBps 00m00s
===> Fetching all distfiles required by libtool-2.4.6 for building
===>  Extracting for libtool-2.4.6
=> SHA256 Checksum OK for libtool-2.4.6.tar.xz.
===>  Patching for libtool-2.4.6
===>   libtool-2.4.6 depends on executable: gm4 - found
===>   libtool-2.4.6 depends on executable: gmake - found
===>   libtool-2.4.6 depends on executable: makeinfo - not found
===>   texinfo-6.5,1 depends on executable: help2man - not found
===>   help2man-1.47.7 depends on executable: gmake - found
===>   help2man-1.47.7 depends on package: perl5>=5.26<5.27 - found
===>  Configuring for help2man-1.47.7
env: ./configure: No such file or directory
===>  Script "configure" failed unexpectedly.
Please report the problem to sunpoet@FreeBSD.org [maintainer] and attach the
"/usr/obj/usr/ports/misc/help2man/work/help2man-1.47.7/config.log" including
the output of the failure of your make command. Also, it might be a good idea
to provide an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1

Stop.
make[5]: stopped in /usr/ports/misc/help2man
*** Error code 1

Stop.
make[4]: stopped in /usr/ports/print/texinfo
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/devel/libtool
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/devel/libuv
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/www/node6
*** Error code 1

Stop.
make: stopped in /usr/ports/net-mgmt/statsd

Obviously, the ports tree seems not to be up to date to the current version of the freebsd Version of the Opnsense and some dependencies are not possible to be resolved/built. Prior to any further telegraf testing, a fix would be required.

Br br
Title: Re: telegraf feature request
Post by: mimugmail on October 14, 2018, 02:40:53 PM
You can request a pkg via github/tools. Then we can go in with it.

After 18.7.5 I'll add the features