OPNsense Forum
English Forums => General Discussion => Topic started by: networkguy on January 21, 2018, 12:17:05 am
-
Is there a way within OPNsense to track individual users historical usage. I would like to be able to view a list of clients and show when they were connected and bandwidth used. Its nice to know data trends on devices on my network. Something like the graphs shown under the interface tab in insight but per client. Would be cool to have a quick visual of user traffic usage. Maybe be able to toggle between ip and hostname like what is on the traffic graphs. Actually, the client usage in the traffic graphs would work great if this was always being captured and data stored to view anytime.
-
You can send your netflow output to an external tool with more features. Solarwinds is one of the more popular ones: https://www.solarwinds.com/free-tools/real-time-netflow-analyzer
Bart...
-
Solarwinds? Name of the project is chosen.. eeehmmm.. unfelicitous
https://en.wikipedia.org/wiki/Stellar_Wind
:-S
-
Thanks for the info. I was looking to see if there is a way to do it on box with a package like bandwidthd. I have been thinking of pushing off firewall logs to splunk, I believe it supports up to 500 MB per day of logging for free. I may try seeing if I could get a net flow dashboard working on there as well.
-
We do have a package for bandwidthd, but no GUI plugin component so it needs to be configured manually.
# pkg add bandwidthd
Cheers,
Franco
-
I don't suppose you have any pointers on configuring this manually?
-
I got bandwidthd to work by installing manually like this. You will need basic command line experience to be able to do this.
1) pkg install bandwidthd
2) Change 'dev' in /usr/local/bandwidthd/etc/bandwidthd.conf to be the interface you want to monitor, for me it was hn0. Usually this will be your LAN interface, you can check the interface names in the Interface->Assignments in the OPNSense UI.
3) Link /usr/local/bandwidthd/htdocs/ to /usr/local/www (I used the command 'ln -s /usr/local/bandwidthd/htdocs/ bandwidthd'). This step I'm not sure if its the correct way to really do this, I'm assuming an update to opnsense in the future might remove this link. Franco I'd love to hear what is the best way to link the bandwidth html docs to the www dir.
4) Then you can start the bandwidthd process by running '/usr/local/etc/rc.d/bandwidthd onestart'. I'm not sure what the correct way to make this startup on boot automatically. Franco again be great if you could let us know what is the ideal way to make it start automatically?
5) Now you can view the graphs at http://192.168.1.X/bandwidthd (it will take a few minutes for them to generate).
Good Luck : ) I have Bandwidthd and ntopng now running nicely, now I can see the bandwidth usage with urls (ntopng) and also over the last week/month (bandwidthd).
-
sub
-
Hi, i am trying to install bandwidthd on my opnsense. I have followed the steps from above but not able to make it work.
Installed the package, changed the interface in config file, link the htdocs folder to bandwidthd folder in www folder, started the service and not working. It takes me to opnsense page which says "page not found."
My login was on different port so i changed it to http port 80. Any settings i am missing ??
my lan is re0 so device in config should be (dev re0) or (dev "re0") ??
Do i need to restart web service as well, i do not find any service to restart, kindly help in this as well.
-
Why dont you use ntopng?
-
I use ntopng and i love it as it shows total bandwidth used, currently active downloader, along with some other great features.
But it do not save the data for previous / old records. I have enabled the cache and changed the time to different time / days. But only thing it shows is active devices. If a device goes offline or firewall reboot then all the data is gone, deleted / removed by software.
I need to provide the daily report of total bandwidth used by the devices in network. I used bandwidthd before in PFSense and i know it will solve this issue.
Still if you know any way around in ntogng to save data and to get past records for report on daily basis then i am good to go with ntopng.
-
It shouldn't be hard to build a plugin for it, but the UI looks really creepy :)
-
I really don't mind the graphics. All i care about i the data / status i will provide me..
I am not a programming guy by any means so If you can build the plugin, will be grateful..
-
Solarwinds? Name of the project is chosen.. eeehmmm.. unfelicitous
https://en.wikipedia.org/wiki/Stellar_Wind
:-S
hot :-X
-
@tillsense. I really do not understand what you are trying to say.
-
I use ntopng and i love it as it shows total bandwidth used, currently active downloader, along with some other great features.
But it do not save the data for previous / old records. I have enabled the cache and changed the time to different time / days. But only thing it shows is active devices. If a device goes offline or firewall reboot then all the data is gone, deleted / removed by software.
I need to provide the daily report of total bandwidth used by the devices in network. I used bandwidthd before in PFSense and i know it will solve this issue.
Still if you know any way around in ntogng to save data and to get past records for report on daily basis then i am good to go with ntopng.
i have the same issue
-
For daily or monthly stats vnstat-plugin?
-
hey guys, i just stumbled on this thread while having the same problem, wanting to use bandwidthd. and also followed them directions by 'dedman'. and also had it not work... BUT...
I figured it out! and got it working just fine, so if anyone still cares about doing this, here it is..
what needs changed is step #3, he had:
ln -s /usr/local/bandwidthd/htdocs/ bandwidthd
it should be :
ln -s /usr/local/bandwidthd/htdocs/ /usr/local/www/bandwidthd
-----
you can see why it wouldn't work. it was running and making stats but putting the html output in the ./bandwidthd folder off whatever folder you were in at the time you ran the command...
so there it is.,
Also i figured out how to autostart the bandwidthd service at startup.
Add a file here:
/usr/local/etc/rc.syshook.d/start
name it something like '91-bwdstartup'
then put these contents in that file you just created:
#!/bin/sh
echo -n "Starting BandwidthD..."
/usr/local/etc/rc.d/bandwidthd onestart
-------------
welp hope this helps someone....
good luck!
-R
-
Is bandwidthd really still a thing? Wasnt updated since when? 2016?
-
Is bandwidthd still the best option for tracking data usage per device per month? I've seen vnstat metnioned too, but I am not sure if it provides usage per device. I have consistently been going over the Xfinity 1.2 TB data cap, and I would like to have a better idea of the main culprits. Thanks.
-
I run ntopng and store my timeseries data in InfluxDB for easy reporting/graphing in Grafana for client bandwidth usage statistics as well as application/category statistics. I only store the data for 2 weeks but could easily keep it for 30+ days if needed. Attached is a screenshot from my Grafana dashboard for the last 7 days for my ntopng data.
I also send OPNsense metrics to InfluxDB via Telegraf and I use Packetbeats to monitor DNS traffic. It makes for a pretty comprehensive dashboard for my home network. I run this all on a Raspberry Pi 4 with a mirror port since I didn't want to interfere with the firewall except for the Telegraf agent. I am assuming it could all be run the firewall itself as well using mimugmail's repository for InfluxDB & Grafana.
-
Grafana Dashboard please :)
-
Attached is my Grafana dashboard for the OPNsense Telegraf metrics and ntopng timeseries. It has two InfluxDB datasource variables to point to for OPNsense telegraf and ntopng databases. I stripped out all panels that require the custom Telegraf things I do for DHCP and DNS/Packetbeats.
For ntopng, I have the following timeseries settings:
- Resolution set to 1 minute
- Interface Layer-7 applications set to Both
- Host Timeseries set to Full
- Host Layer-7 applications set to Both
For the OPNsense CPU temperature, I manually added the following to the telegraf script:
[[inputs.exec]]
commands = ["/sbin/sysctl dev.cpu | grep temperature | sed 's/[a-z\.]*/systemp cpu/;s/\.[a-z]*\: /=/;s/.$//'"]
timeout = "5s"
data_format = "influx"
-
Thx for sharing, I will have a look (with Plugins-only :) )
-
Thanks rman50 and mimugmail! This seems a bit over my head at the moment. I do have a HP 730t for Opensense, and I have a couple of Raspberry Pi 4's so I could attempt this since it seems pretty cool to have that level of visibility into usage.
I'm currently on the Xfinity XB7 in router mode since the unlimited data plan is $25 with their gateway and $30 if I use my own modem, but I plan on spinning up OPNsense and putting the XB7 in bridge mode which I've done many times before without issue.
So I guess bandwidthd is the best option outside of using ntopng piped to a Raspberry Pi?
-
I am running the latest OPNsense, and I setup bandwidthd based on the previous comments in this topic. I will give this a try for a while to see if it provides the information I'm looking for, but on first glance it looks good. Thanks again!
-
I have been using bandwidthd for the last couple days, and it does great on ipv4, but it completely misses all of my ipv6 traffic. I'm assuming I will need to use ntopng to capture both ipv4 and ipv6.
I'm going to look into ntopng tutorials next. Would ya'll recommend I store the ntopng data on the OPNsense firewall since I only plan on using it for a couple of months or should I send it to a raspberry pi 4? I have over 200GB storage and 8GB of memory on the firewall (HP 730t) so I could probably use that if I wanted, but I also don't want to risk slowing down my 1200-1400 mbps connection. I'm not running any IDS/IPS.
-
ntopng does support both ipv4 and ipv6. influxdb is very efficient at storing timeseries data since that is all it does. Also, ntopng provides several options for what data to retain and how often to write it out (every minute or every 5 minutes). Other than the timeseries data, ntopng maintains everything else in memory. I just reset my environment but I was using almost no disk space to store 2 weeks worth of data and I enable most of the timeseries details. I doubt you would have any issue running ntopng, influxdb & grafana (if you want your own custom dashboards which I recommend) on your firewall with those specs.
-
Thanks so much rman50!!
Hopefully, I will have some time over the weekend to work on getting this setup on my HP 730t running OPNsense. Am I understanding correctly that the json file you provided I could use with my grafana installation?
I did notice that there is an option for enabling nProbe in ntopng. Do you think this is something that I look into or just leave it disabled? It sounds interesting from a data collection standpoint, but I don't think I would use the nDPI function.
I'm also curious in how you are sending OPNsense metrics to influxdb via telegraph. I'm going to look into that too.
Thanks again! I'm really glad that I decided to go with OPNsense based on the friendly and knowledgeable people like you in this forum and the options that OPNsense provides. :)
-
I don't use nProbe. It isn't required for the client statistics. Also I think it requires a license.
You should be able to import my dashboard json into grafana and then customize it as needed.
There is a telegraf plugin for opnsense which makes it simple to send the firewall metrics to influx.
-
I got it running quite quickly with ntop :)
-
Hey guys, new here.
I was just setting up the telegraph-influxdb-grafana setup but for some reason im unable to send metrics to influxdb using telegraf. Im unable to find the opnsense telegraf plugin on influxdb.
Can anyone help me on the same?
Thanks.