Hello all,
After a reboot, insight stopt working. I get "No Data Available"
I'm also sending netflow data to prtg that is still working fine.
I have try-ed to populate the data in the netflow tab again.
No messages in the syslog, any ideas where to look ?
Syd
Just upgraded to 16.1.17 problem is still there.
Anybody ??
Are you using a nano image and/or periodic backups?
Hello Franco,
Running:
OPNsense 16.1.17-amd64
FreeBSD 10.2-RELEASE-p18
OpenSSL 1.0.2h 3 May 2016
On esxi 5.5 started with config of monowall in early time of opnsense and updated along the way.
No scheduled backups
Hi Syd,
Ok, there's an unconditional backup task at reboot time, too. Just making sure this one is affected too.
What happens is that the database gets copied during an inconsistent state and can't be recovered after boot.
I had an install where this undid itself after a while, but that's obviously not the right fix.
Here's a dump of my current files, I'm curious to know how yours looks:
# ls -lah /var/netflow/
total 103036
drwxr-x--- 2 root wheel 768B Jun 16 20:24 .
drwxr-xr-x 16 root wheel 960B Jun 16 03:01 ..
-rw-r----- 1 root wheel 76K Jun 16 20:24 dst_port_000300.sqlite
-rw-r----- 1 root wheel 144K Jun 16 20:24 dst_port_003600.sqlite
-rw-r----- 1 root wheel 9.4M Jun 16 20:24 dst_port_086400.sqlite
-rw-r----- 1 root wheel 1.8M Jun 16 20:24 interface_000030.sqlite
-rw-r----- 1 root wheel 1.3M Jun 16 20:24 interface_000300.sqlite
-rw-r----- 1 root wheel 416K Jun 16 20:24 interface_003600.sqlite
-rw-r----- 1 root wheel 40K Jun 16 20:24 interface_086400.sqlite
-rw-r----- 1 root wheel 8.0K Jun 16 20:24 metadata.sqlite
-rw-r----- 1 root wheel 284K Jun 16 20:24 src_addr_000300.sqlite
-rw-r----- 1 root wheel 936K Jun 16 20:24 src_addr_003600.sqlite
-rw-r----- 1 root wheel 16M Jun 16 20:24 src_addr_086400.sqlite
-rw-r----- 1 root wheel 70M Jun 16 20:24 src_addr_details_086400.sqlite
Cheers,
Franco
Hey franco,
First thnx for the help,
total 72172
drwxr-x--- 2 root wheel 512B Jun 12 20:43 .
drwxr-xr-x 29 root wheel 512B Jun 1 06:43 ..
-rw-r----- 1 root wheel 196K Jun 5 22:06 dst_port_000300.sqlite
-rw-r----- 1 root wheel 432K Jun 5 22:06 dst_port_003600.sqlite
-rw-r----- 1 root wheel 748K Jun 5 22:06 dst_port_086400.sqlite
-rw-r----- 1 root wheel 2.1M Jun 5 22:13 interface_000030.sqlite
-rw-r----- 1 root wheel 1.5M Jun 5 22:06 interface_000300.sqlite
-rw-r----- 1 root wheel 252K Jun 5 22:06 interface_003600.sqlite
-rw-r----- 1 root wheel 28K Jun 5 22:06 interface_086400.sqlite
-rw-r----- 1 root wheel 8.0K Jun 5 22:06 metadata.sqlite
-rw-r----- 1 root wheel 616K Jun 5 22:06 src_addr_000300.sqlite
-rw-r----- 1 root wheel 2.9M Jun 5 22:06 src_addr_003600.sqlite
-rw-r----- 1 root wheel 9.2M Jun 5 22:06 src_addr_086400.sqlite
-rw-r----- 1 root wheel 52M Jun 5 22:06 src_addr_details_086400.sqlite
is my /var/netflow
hello;
I have the same problem too
Hi,
The files that where in /var/netflow didn't change from the time the problem started, i turned netflow of removed all the files and turned it back on. It recreated the files again, but it still doesn't work.
This is the new dir:
total 140
drwxr-x--- 2 root wheel 1.0K Jun 18 10:25 .
drwxr-xr-x 29 root wheel 512B Jun 1 06:43 ..
-rw-r----- 1 root wheel 12K Jun 18 10:11 dst_port_000300.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 dst_port_003600.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 dst_port_086400.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 interface_000030.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 interface_000300.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 interface_003600.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 interface_086400.sqlite
-rw-r----- 1 root wheel 0B Jun 18 10:11 metadata.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 src_addr_000300.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 src_addr_003600.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 src_addr_086400.sqlite
-rw-r----- 1 root wheel 12K Jun 18 10:11 src_addr_details_086400.sqlite
Any clues ?
Hi syd,
Your database files and metadata was definitely trashed. There is no data to be found. Can you give the following output too on your mounted filesystems?
# df -h
There is also netflow cache file in /var/log that may be corrupted too. The reporting database pulls the standard netflow data from there... Maybe deleting this file if seemingly damaged would be a better starting point.
# ls -lah /var/log/flowd.*
So far I haven't been able to reproduce the problem anymore.
The frontend needs a while time to collect initial data, but it should really jump back to life after 10 minutes.
Thanks,
Franco
Hey Franco,
root@mono:~ # ls -lah /var/log/flowd.*
-rw------- 1 root wheel 2.8G Jun 21 22:32 /var/log/flowd.log
-rw------- 1 root wheel 11M Jun 5 19:58 /var/log/flowd.log.000001
-rw------- 1 root wheel 11M Jun 5 16:28 /var/log/flowd.log.000002
-rw------- 1 root wheel 11M Jun 5 12:51 /var/log/flowd.log.000003
-rw------- 1 root wheel 11M Jun 5 10:17 /var/log/flowd.log.000004
-rw------- 1 root wheel 11M Jun 5 06:15 /var/log/flowd.log.000005
-rw------- 1 root wheel 11M Jun 5 01:15 /var/log/flowd.log.000006
-rw------- 1 root wheel 11M Jun 4 21:41 /var/log/flowd.log.000007
-rw------- 1 root wheel 11M Jun 4 18:02 /var/log/flowd.log.000008
-rw------- 1 root wheel 11M Jun 4 14:38 /var/log/flowd.log.000009
-rw------- 1 root wheel 11M Jun 4 11:58 /var/log/flowd.log.000010
root@mono:~ # df -h
Filesystem Size Used Avail Capacity Mounted on"+
/dev/da0s1a 31G 4.3G 24G 15% /
devfs 1.0K 1.0K 0B 100% /dev
devfs 1.0K 1.0K 0B 100% /var/dhcpd/dev+
Will get rid of -rw------- 1 root wheel 2.8G Jun 21 22:32 /var/log/flowd.log
and see if it recreats it.
Grtz
Syd
Quick update,
It works again
-rw------- 1 root wheel 3.6M Jun 22 13:51 flowd.log
Is recreated and updating.
@ Franco If you want i could send you the faulty flowd.log, it is quit big 2.7 gig.
Thanx for pointing me in the right dir.
Grtz
Syd
Hi Syd,
If we could pull that trade off it would be nice. This is a huge file and the likely cause of the graph not regenerating. Thanks for your help so far.
Do you have a server where you could push this file to grab privately?
Cheers,
Franco