OPNsense Forum

English Forums => Development and Code Review => Topic started by: tillsense on December 28, 2016, 08:08:01 pm

Title: [SOLVED] Netflow Error on Reboot
Post by: tillsense on December 28, 2016, 08:08:01 pm
hi,
on a boot/reboot, these messages are displayed.

Quote
configd.py: error generating template OPNsense/Netflow : Traceback (most recent call last): File "/usr/local/opnsense/service/modules/template.py", line 301, in generate for filename in self._generate(template_name, create_directory): File "/usr/local/opnsense/service/modules/template.py", line 236, in _generate raise Exception("%s %s %s" % (module_name, template_filename, render_exception)) Exception: OPNsense/Netflow OPNsense/Netflow/netflow.conf 'collections.OrderedDict object' has no attribute 'egress_only'

cheers till

--------------------------------------------------------------------
OPNsense 17.1.b_73-amd64
FreeBSD 11.0-RELEASE-p5
OpenSSL 1.0.2j 26 Sep 2016
Title: Re: Netflow Error on Reboot
Post by: AdSchellevis on December 30, 2016, 05:47:53 pm
Hi till,

It looks like a bug, when no "egress_only" interfaces are provided it fails generating the template.

You should be able to fix this using the following patch https://github.com/opnsense/core/commit/5f3936ab3696f384f29bb5ee1290f4c4c4c1572a (https://github.com/opnsense/core/commit/5f3936ab3696f384f29bb5ee1290f4c4c4c1572a)

On the command line, just run the following command to patch your version.
Code: [Select]
opnsense-patch 5f3936ab36

Best regards,

Ad
Title: Re: Netflow Error on Reboot
Post by: tillsense on December 30, 2016, 07:01:14 pm
Hi Ad,

that looks good! thanks

cheers till

-------------------------------------
OPNsense 17.1.b_74-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: tillsense on January 02, 2017, 07:45:19 pm
Hi Ad,

the file (command)

Code: [Select]
/usr/local/bin/python2.7
/usr/local/opnsense/scripts/netflow/flowd_aggregate.py

caused since the changes quite high cpu last (eg 46.39% idle client usage)

cheers till

-------------------------------------
OPNsense 17.1.b_82-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: AdSchellevis on January 02, 2017, 07:55:03 pm
Hi till,

Then it probably is catching up after some time of not being active.
You can always remove the unprocessed entries by removing all of /var/log/flowd.log* and restart both flowd and the aggregator.

If netflow is active and the aggregator isn't running, it will parse the missed items when starting again, which can be quite heavy (and usually gets back to normal load when done).

Best regards,

Ad
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: tillsense on January 03, 2017, 06:33:54 pm
Hi Ad,

that was the problem (/var/log/flowd.log*). Thanks for the quick help!

cheers till

-------------------------------------
OPNsense 17.1.b_86-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: tillsense on January 10, 2017, 08:46:09 pm
Hi Ad,

i can confirm this https://forum.opnsense.org/index.php?topic=4235.0 (https://forum.opnsense.org/index.php?topic=4235.0) under 17.1 there must still be a problem.
(cpu last is also there again)

cheers till


-------------------------------------------
OPNsense 17.1.b_105-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: AdSchellevis on January 11, 2017, 08:43:10 am
Hi till,

If the aggregator is running and there is traffic in the flow log there probably isn't really an issue.... netflow aggregation is a pretty consuming job, if the aggregation daemons dies, you could try to run in it a console and check its output.

Regards,

Ad
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: tillsense on January 21, 2017, 08:17:48 pm
Ok, back to Python 2.7.12: (https://forum.opnsense.org/index.php?topic=4295.0 (https://forum.opnsense.org/index.php?topic=4295.0)) (e.g. 16.7.13: https://forum.opnsense.org/index.php?topic=4235.0 (https://forum.opnsense.org/index.php?topic=4235.0))

Code: [Select]
pkg add -f https://pkg.opnsense.org/FreeBSD:11:amd64/17.1/MINT/17.1.b/OpenSSL/All/python27-2.7.12.txzseems to be the first solution.

cheers till

-------------------------------------------
OPNsense 17.1.r-amd64
FreeBSD 11.0-RELEASE-p7
OpenSSL 1.0.2j 26 Sep 2016
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: tillsense on February 22, 2017, 07:51:07 pm
Hi,

wow. the first reboot with 17.1.2 on dev apu (1d) shows no more cpu consuming job for aggregate. realtek driver?

cheers till
Title: Re: [SOLVED] Netflow Error on Reboot
Post by: franco on February 22, 2017, 08:13:35 pm
Hi Till,

Likely candidate, yet still hard to believe. In any case, happy about the development here.


Cheers,
Franco