OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • English Forums »
  • General Discussion »
  • flowd_aggregate is not running?
« previous next »
  • Print
Pages: [1]

Author Topic: flowd_aggregate is not running?  (Read 10190 times)

onazari

  • Newbie
  • *
  • Posts: 17
  • Karma: 0
    • View Profile
flowd_aggregate is not running?
« on: July 10, 2016, 10:51:40 am »
Hi;
flowd_aggregate is not running, How to fix it?
Logged

onazari

  • Newbie
  • *
  • Posts: 17
  • Karma: 0
    • View Profile
Re: flowd_aggregate is not running?
« Reply #1 on: July 10, 2016, 10:58:45 am »
flowd_aggregate output:
# service flowd_aggregate start
Starting flowd_aggregate.
# service flowd_aggregate start
flowd_aggregate already running?  (pid=74084).
 # service flowd_aggregate status
flowd_aggregate is not running.
 # service flowd_aggregate status
flowd_aggregate is not running.
Logged

AdSchellevis

  • Administrator
  • Hero Member
  • *****
  • Posts: 849
  • Karma: 163
    • View Profile
Re: flowd_aggregate is not running?
« Reply #2 on: July 10, 2016, 02:34:08 pm »
Hi,

can you try to run this in a console and paste the output here?

Code: [Select]
/usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
Best regards,

Ad
Logged

onazari

  • Newbie
  • *
  • Posts: 17
  • Karma: 0
    • View Profile
Re: flowd_aggregate is not running?
« Reply #3 on: July 11, 2016, 05:43:11 am »
Quote from: AdSchellevis on July 10, 2016, 02:34:08 pm
Hi,

can you try to run this in a console and paste the output here?

Code: [Select]
/usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
Best regards,

Ad
Hi, Tanks For Your Replay


# /usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
# /usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
# /usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
 Nothing output

Logged

AdSchellevis

  • Administrator
  • Hero Member
  • *****
  • Posts: 849
  • Karma: 163
    • View Profile
Re: flowd_aggregate is not running?
« Reply #4 on: July 11, 2016, 08:44:58 am »
ok, that's not much  :)

can you try to grep your syslog?

Code: [Select]
clog /var/log/system.log | grep flowd

Logged

onazari

  • Newbie
  • *
  • Posts: 17
  • Karma: 0
    • View Profile
Re: flowd_aggregate is not running?
« Reply #5 on: July 11, 2016, 09:57:24 am »
Quote from: AdSchellevis on July 11, 2016, 08:44:58 am
ok, that's not much  :)

can you try to grep your syslog?

Code: [Select]
clog /var/log/system.log | grep flowd


Output is:
Code: [Select]
# clog /var/log/system.log | grep flowd
Jul  3 09:00:34 OPNsense root: /usr/local/etc/rc.d/flowd: WARNING: failed to start flowd
Jul  9 16:32:14 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 11:49:45 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 12:03:40 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 12:04:56 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:18:38 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:18:53 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:19:10 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:21:57 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:22:00 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:22:07 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:23:52 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:24:37 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:24:53 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 10 13:24:56 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 07:19:24 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 08:09:32 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 08:11:11 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 08:11:14 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 11:35:04 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 11:35:07 bop root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate
Jul 11 11:35:08 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 11:35:15 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small
Jul 11 11:35:21 bop flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last):   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 145, in run     aggregate_flowd(do_vacuum)   File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 67, in aggregate_flowd     for flow_record in parse_flow(prev_recv):   File "/usr/local/opnsense/scripts/netflow/lib/parse.py", line 92, in parse_flow     for flow in flog: ValueError: store_read_flow: Internal error: flow buffer too small

Logged

AdSchellevis

  • Administrator
  • Hero Member
  • *****
  • Posts: 849
  • Karma: 163
    • View Profile
Re: flowd_aggregate is not running?
« Reply #6 on: July 11, 2016, 10:27:19 am »
It looks like a corruption in your flowd log files, you could remove all collected data and start over.


Code: [Select]
rm /var/log/flowd.log*
rm /var/netflow/*.sqlite

The /var/netflow/ contains the already parsed data (if any), you could first try to keep that one, and if that doesn't work remove those too.
Logged

franco

  • Administrator
  • Hero Member
  • *****
  • Posts: 13616
  • Karma: 1170
    • View Profile
Re: flowd_aggregate is not running?
« Reply #7 on: July 11, 2016, 11:00:50 am »
We do need a GUI reset for this, I've recorded this here: https://github.com/opnsense/core/issues/983
Logged

Supermule

  • Full Member
  • ***
  • Posts: 223
  • Karma: 15
    • View Profile
Re: flowd_aggregate is not running?
« Reply #8 on: March 30, 2020, 07:53:16 pm »
Same error today

Mar 30 19:39:54    /flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last): File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 160, in run aggregate_flowd(self.config, do_vacuum) File…

Version 20.1.3
Logged

adiz0r

  • Newbie
  • *
  • Posts: 12
  • Karma: 0
  • I'm a Hungarian *nix engineer trapped in Vienna.
    • View Profile
Re: flowd_aggregate is not running?
« Reply #9 on: May 15, 2020, 10:00:07 pm »
flowd_aggregate regularly dies. I had to create a monit task for it to restart and keep it alive.
Logged

fog

  • Newbie
  • *
  • Posts: 22
  • Karma: 1
    • View Profile
Re: flowd_aggregate is not running?
« Reply #10 on: July 20, 2020, 03:39:12 pm »
Quote
I had to create a monit task for it to restart and keep it alive.
I have the same problem. The flowd_aggregate service stop after 1-7 days.
My working solution: In Services: Monit: Settings create new service:

Enable service checks: checked
Name: flowd_aggregate_monitor
Type: Process
PID file: /var/run/flowd_aggregate.pid
Start: /usr/sbin/service flowd_aggregate start
Stop: /usr/sbin/service flowd_aggregate stop
Tests: Nothing selected
Depends: Nothing selected


If the email settings are correct set under Services: Monit: Settings -> General settings and Alert Settings mails are sent:

Does not exist Service flowd_aggregate_monitor

   Date:        Sun, 19 Jul 2020 06:00:02
   Action:      restart
   Host:        ****
   Description: process is not running

Your faithful employee,
Monit


Exists Service flowd_aggregate_monitor

   Date:        Sun, 19 Jul 2020 06:02:10
   Action:      alert
   Host:        ****
   Description: process is running with pid 58260

Your faithful employee,
Monit




Logged

GuruLee

  • Newbie
  • *
  • Posts: 2
  • Karma: 0
    • View Profile
Re: flowd_aggregate is not running?
« Reply #11 on: November 06, 2020, 12:05:58 pm »
Quote from: franco on July 11, 2016, 11:00:50 am
We do need a GUI reset for this, I've recorded this here: https://github.com/opnsense/core/issues/983
Thank you, this fixed the no insight data for me.
Logged

  • Print
Pages: [1]
« previous next »
  • OPNsense Forum »
  • English Forums »
  • General Discussion »
  • flowd_aggregate is not running?
 

OPNsense is an OSS project © Deciso B.V. 2015 - 2023 All rights reserved
  • SMF 2.0.19 | SMF © 2021, Simple Machines
    Privacy Policy
    | XHTML | RSS | WAP2