[SOLVED] Netflow Error on Reboot

[tillsense]

hi,
on a boot/reboot, these messages are displayed.

configd.py: error generating template OPNsense/Netflow : Traceback (most recent call last): File "/usr/local/opnsense/service/modules/template.py", line 301, in generate for filename in self._generate(template_name, create_directory): File "/usr/local/opnsense/service/modules/template.py", line 236, in _generate raise Exception("%s %s %s" % (module_name, template_filename, render_exception)) Exception: OPNsense/Netflow OPNsense/Netflow/netflow.conf 'collections.OrderedDict object' has no attribute 'egress_only'

cheers till

--------------------------------------------------------------------
OPNsense 17.1.b_73-amd64
FreeBSD 11.0-RELEASE-p5
OpenSSL 1.0.2j 26 Sep 2016

[AdSchellevis]

Hi till,

It looks like a bug, when no "egress_only" interfaces are provided it fails generating the template.

You should be able to fix this using the following patch https://github.com/opnsense/core/commit/5f3936ab3696f384f29bb5ee1290f4c4c4c1572a

On the command line, just run the following command to patch your version.

Code: [Select]

opnsense-patch 5f3936ab36

Best regards,

Ad

[tillsense]

Hi Ad,

that looks good! thanks

cheers till

-------------------------------------
OPNsense 17.1.b_74-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016

[tillsense]

Hi Ad,

the file (command)

Code: [Select]

/usr/local/bin/python2.7
/usr/local/opnsense/scripts/netflow/flowd_aggregate.py
caused since the changes quite high cpu last (eg 46.39% idle client usage)

cheers till

-------------------------------------
OPNsense 17.1.b_82-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016

[AdSchellevis]

Hi till,

Then it probably is catching up after some time of not being active.
You can always remove the unprocessed entries by removing all of /var/log/flowd.log* and restart both flowd and the aggregator.

If netflow is active and the aggregator isn't running, it will parse the missed items when starting again, which can be quite heavy (and usually gets back to normal load when done).

Best regards,

Ad

[tillsense]

Hi Ad,

that was the problem (/var/log/flowd.log*). Thanks for the quick help!

cheers till

-------------------------------------
OPNsense 17.1.b_86-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016

[tillsense]

Hi Ad,

i can confirm this https://forum.opnsense.org/index.php?topic=4235.0 under 17.1 there must still be a problem.
(cpu last is also there again)

cheers till


-------------------------------------------
OPNsense 17.1.b_105-amd64
FreeBSD 11.0-RELEASE-p6
OpenSSL 1.0.2j 26 Sep 2016

[AdSchellevis]

Hi till,

If the aggregator is running and there is traffic in the flow log there probably isn't really an issue.... netflow aggregation is a pretty consuming job, if the aggregation daemons dies, you could try to run in it a console and check its output.

Regards,

Ad

[tillsense]

Ok, back to Python 2.7.12: (https://forum.opnsense.org/index.php?topic=4295.0) (e.g. 16.7.13: https://forum.opnsense.org/index.php?topic=4235.0)

Code: [Select]

pkg add -f https://pkg.opnsense.org/FreeBSD:11:amd64/17.1/MINT/17.1.b/OpenSSL/All/python27-2.7.12.txzseems to be the first solution.

cheers till

-------------------------------------------
OPNsense 17.1.r-amd64
FreeBSD 11.0-RELEASE-p7
OpenSSL 1.0.2j 26 Sep 2016

[tillsense]

Hi,

wow. the first reboot with 17.1.2 on dev apu (1d) shows no more cpu consuming job for aggregate. realtek driver?

cheers till

[franco]

Hi Till,

Likely candidate, yet still hard to believe. In any case, happy about the development here.


Cheers,
Franco