Netflow Insight not showing any data

[interfaSys]

I've followed the steps in the doc, but left the destination blank.
I added wan, a gateway and a vlan as a test
I left the version at v9, even though IPv6 is disabled since it's not entirely disabled according to the list of sockets I see.

If I go to the "Insight" page, the graphs are empty: "no data available"

The logs are spammed with

May 19 17:48:00   configd.py: [f300521e-863a-4cd3-97fa-0bad73b2ca06] request netflow data aggregator metadata
May 19 17:47:59   configd.py: [ef199544-de3e-46c2-8093-d3e1a46f1cac] request netflow data aggregator top usage for FlowInterfaceTotals
May 19 17:47:59   configd.py: [df958d68-9646-4f2b-a8b8-35cab2fbff88] request netflow data aggregator top usage for FlowInterfaceTotals
May 19 17:47:59   configd.py: [2a413224-0022-48c8-a8d8-de948bce05ee] request netflow data aggregator top usage for FlowSourceAddrTotals
May 19 17:47:59   configd.py: [4d36559a-6244-43c8-a5a0-8492686bb371] request netflow data aggregator top usage for FlowDstPortTotals
May 19 17:47:59   configd.py: [0f664e35-f9e2-4273-880a-bd80d66e5230] request netflow data aggregator timeseries for FlowInterfaceTotals

[interfaSys]

If I go to the Netflow->Cache tab, I can see that things are being tracked as I can see the number of sources, destinations and packets for each selected interface.

[AdSchellevis]

what does this output?

Code: [Select]

flowd-reader /var/log/flowd.log


[interfaSys]

what does this output?

Tons of info

One example

Code: [Select]

FLOW recv_time 2016-05-19T18:01:20.186513 proto 6 tcpflags 1b tos 00 agent [127.0.0.1] src [1.2.3.4]:443 dst [10.2.3.4]:61170 packets 6 octets 1605

[AdSchellevis]

ok, that's good... netflow data is flowing

then this:

Code: [Select]

service flowd_aggregate stop
/usr/local/opnsense/scripts/netflow/flowd_aggregate.py console

the second statement should "hang" while processing data, <ctrl><c> after a minute to exit.

[interfaSys]

I just looked at the logs and saw this, so maybe the config is now conplete due to the crash:

Code: [Select]

May 19 17:51:13 Local configd.py: [e3e26f3e-a431-460b-8257-047405f92b68] generate template OPNsense.Netflow
May 19 17:51:15 Local configd.py: generate template container OPNsense.Netflow
May 19 17:51:15 Local configd.py: [5bfc78ab-6726-4131-b725-0eb3c9a00ae1] stop netflow
May 19 17:51:15 Local configd.py: [d51a61df-2435-4bf9-8fb4-0b96872efc8d] start netflow
May 19 17:51:18 Local configd.py: [c73d8942-975d-474e-a6da-f09e000fbd6e] request status of netflow collector
May 19 17:51:18 Local configd.py: [15c4e875-079b-44ee-8356-7dba9e4c4311] restart netflow data aggregator
May 19 17:51:21 Local python2.7: stack overflow detected; terminated
May 19 17:51:21 Local kernel: pid 67136 (python2.7), uid 0: exited on signal 6 (core dumped)

[interfaSys]

Ah, seems the aggregator segfaults

Code: [Select]

# /usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
Abort trap (core dumped)

[AdSchellevis]

might be kernel/hardware related... I haven't seen it on one of our boxes.

can you try this:

Code: [Select]

root@OPNsense:~/core # python2.7
Python 2.7.11 (default, Apr  6 2016, 12:15:49)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import flowd
>>> flog = flowd.FlowLog('/var/log/flowd.log')
>>> for flow in flog:
...   print flow
...   break
...
<flowd.Flow object at 0x2faab2449c0>

the <flow.Flow..> is output, the print and break needs two spaces in front..

[interfaSys]

Here is my output:

Code: [Select]

# python2.7
Python 2.7.11 (default, Apr 27 2016, 03:16:54)
[GCC 4.2.1 Compatible Clang 3.7.1 (tags/RELEASE_371/final)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import flowd
>>> flog = flowd.FlowLog('/var/log/flowd.log')
>>> for flow in flog:
...   print flow
...   break
...
<flowd.Flow object at 0x8006cd8b0>
>>>
Hardware is N3150. Also I'm using Clang 3.7.1.

[AdSchellevis]

Sorry, I can't test that... could you try to install our standard system and retry?

[interfaSys]

Sorry, I can't test that... could you try to install our standard system and retry?

Unfortunately, everything is compiled from source, so I can't easily switch to packages.
I tried gdb, but it didn't give me anything useful

Code: [Select]

Starting program: /usr/local/bin/python2 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py console
Abort trap (core dumped)

Program exited with code 0206.
You can't do that without a process to debug.

[interfaSys]

One lead from the logs above is this:

Code: [Select]

python2.7: stack overflow detected; terminated
I'm using stack protection for all my packages and apparently a buffer overflow is detected.

Additional flags: -D_FORTIFY_SOURCE=2 -fstack-protector-strong --param ssp-buffer-size=4