[SOLVED] Netflow Insight No Data Available

Archive > 16.7 Legacy Series

(1/9) > >>

adcwb:
OPNsense was running ok until yesterday, as of today the Insight page just show "No Data Available".
The log shows the following:

--- Code: ---flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last): File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run aggregate_flowd(do_vacuum) File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 79, in aggregate_flowd stream_agg_object.add(flow_record_cpy) File "/usr/local/opnsense/scripts/netflow/lib/aggregates/source.py", line 105, in add super(FlowSourceAddrDetails, self).add(flow) File "/usr/local/opnsense/scripts/netflow/lib/aggregate.py", line 258, in add self._update_cur.execute(self._update_stmt, flow) DatabaseError: database disk image is malformed
--- End code ---

In the Netflow -> Cache page i can see that it is tracking.
The flowd-reader command seems fine too, outputting the traffic.
Tried restarting the flowd_aggregator service, without success.

Any thoughts?

franco:
Hi there,

Which version are you using? What image flavour did you use?

We do have an auto-repair, but it's not in the release it. You can apply it on top of 16.7.2 using:

# opnsense-patch 5ec2101

The respective commit is:

https://github.com/opnsense/core/commit/5ec2101

If this works we can queue it up for 16.7.3 next week.

Cheers,
Franco

Zapp:
Bumping this.

As per my upgrade to 16.7.3 yesterday (31/8) I now have the No Data Available in my Insight display. 31/8 was ok and have data up until time of upgrade then nothing. 1/9 have NULL entries!

In NetFlow section Cache tab pressing refresh makes the numbers change. In my world that means something is happening but does not get reflected in Insight.

NetFlow Capture is on and Capture local checked, 127.0.0.1:2056 for Destination and v9 for Version.

I'd be happy to supply more information but right now I don't know what. :-\

/Jonas...

franco:
Hi Jonas,

You might want to try this patch we've given a longer test interval:

https://github.com/opnsense/core/commit/5ec2101ac2

You can install and activate it from the command like this

# opnsense-patch 5ec2101ac2
# service flowd_aggregate restart

The graphs should come back to life immediately.

I haven't seen any glitches there in the last month (I'm using the patched opnsense-devel version).

Cheers,
Franco

Zapp:

--- Quote from: franco on September 01, 2016, 02:05:37 pm ---...
You can install and activate it from the command like this

# opnsense-patch 5ec2101ac2
# service flowd_aggregate restart

The graphs should come back to life immediately.
...

--- End quote ---

Sorry, no joy!

--- Code: ---Edit /etc/motd to change this login announcement.

0) Logout 7) Ping host
1) Assign Interfaces 8) Shell
2) Set interface(s) IP address 9) pfTop
3) Reset the root password 10) Filter Logs
4) Reset to factory defaults 11) Restart web interface
5) Power off system 12) Upgrade from console
6) Reboot system 13) Restore a configuration

Enter an option: 8

root@OPNsense:~ # opnsense-patch 5ec2101ac2
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From 5ec2101ac2c67824b94306fe01d1ed97211ea730 Mon Sep 17 00:00:00 2001
|From: Ad Schellevis <ad@opnsense.org>
|Date: Sun, 7 Aug 2016 16:10:19 +0200
|Subject: [PATCH] (insight) check database integrity before start, repair if
| broken
|
|---
| src/opnsense/scripts/netflow/flowd_aggregate.py | 5 +-
| src/opnsense/site-python/sqlite3_helper.py | 69 +++++++++++++++++++++++++
| 2 files changed, 73 insertions(+), 1 deletion(-)
| create mode 100644 src/opnsense/site-python/sqlite3_helper.py
|
|diff --git a/src/opnsense/scripts/netflow/flowd_aggregate.py b/src/opnsense/scripts/netflow/flowd_aggregate.py
|index 8915576..1368046 100755
|--- a/src/opnsense/scripts/netflow/flowd_aggregate.py
|+++ b/src/opnsense/scripts/netflow/flowd_aggregate.py
--------------------------
Patching file opnsense/scripts/netflow/flowd_aggregate.py using Plan A...
Hunk #1 succeeded at 28.
Hunk #2 succeeded at 36.
Hunk #3 succeeded at 130.
Hmm... The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/src/opnsense/site-python/sqlite3_helper.py b/src/opnsense/site-python/sqlite3_helper.py
|new file mode 100644
|index 0000000..6c32f7e
|--- /dev/null
|+++ b/src/opnsense/site-python/sqlite3_helper.py
--------------------------
(Creating file opnsense/site-python/sqlite3_helper.py...)
Patching file opnsense/site-python/sqlite3_helper.py using Plan A...
Empty context always matches.
Hunk #1 succeeded at 1.
done
All patches have been applied successfully. Have a nice day.
root@OPNsense:~ # service flowd_aggregate restart
flowd_aggregate not running? (check /var/run/flowd_aggregate.pid).
Starting flowd_aggregate.
root@OPNsense:~ # service flowd_aggregate status
flowd_aggregate is running as pid 3230.
root@OPNsense:~ #

--- End code ---

Still No Data Available i graphs and still nothing in Details either.

/Jonas...

Navigation

[0] Message Index

[#] Next page

Go to full version