OPNsense Forum

Archive => 17.7 Legacy Series => Topic started by: wizziLalev on November 08, 2017, 09:41:44 pm

Title: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 08, 2017, 09:41:44 pm

Hi all,
Just after updating to the last release I've notice that flowd_aggregate.py process start to use almost 100% of the CPU.

So I start to read different topics here and following is done:
1. All logs are removed and the machine is restarted
2. RRD is disabled
3. The process is being restarted

and still no luck - the cpu usage is still high.

-->screenshot<-- (https://drive.google.com/file/d/1n8aLfLLREmkUCMGttqlCTWASxU25cqQA/view?usp=sharing)

Any ideas are welcome!

Thanks!

Title: Re: flowd_aggregate.py very high cpu usage
Post by: bartjsmit on November 08, 2017, 09:44:25 pm

You can try Reporting, Netflow, clear all interfaces, then add them back in. You will lose your historic netflow information under Insight.

Bart...

Title: Re: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 10, 2017, 09:49:50 pm

Hi Bart,
Thanks for advice, but the situation is the same, even after new restart.
Any other ideas? The same process is using ~100% of my cpu again.

Best,
Lalev

Title: Re: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 12, 2017, 10:25:54 am

Any other ideas? Anyone?

Title: Re: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 17, 2017, 01:23:16 pm

up - the issue still persist and I really want to fix it - almost constantly my cpu is under full load and this slows the entire firewall and my network also.

I can provide more information if it is needed - but I need a clue for what to look...

Title: Re: flowd_aggregate.py very high cpu usage
Post by: franco on November 17, 2017, 05:22:44 pm

You have neither posted your hardware nor the traffic volume you are pushing... on the offchance that the system simply can’t cope with reporting which isn’t unusual because commercial reporting mostly comes as a separate appliance to not drain the processing power of the operational appliance.


Cheers,
Franco

Title: Re: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 17, 2017, 06:56:33 pm

Hi @franco,

This is my hardware:

Athlon II x4 @3.00Ghz (I know it is old cpu but it still ticks)
8GB RAM DDR3 @1033Mhz
4x1Gbps interfaces
100Mbps internet connection
6 devices connected via switch (5 SBCs + 1AP)
3 wireless connected devices

Pure installation with very simple configuration, nothing major.

The traffic overall is not exceeding 40Mbps because most of the devices are in idle.

Just before latest update everything was perfect 1~2% CPU load, 500~700MB RAM usage and stable network.

I've already try to disconnect everything just to confirm that the connected devices are not causing the problem, but the result stay all the same.

Cheers,
Lalev

Title: Re: flowd_aggregate.py very high cpu usage
Post by: franco on November 18, 2017, 08:04:06 am

Can you paste the output of the following commands?

# ls -lah /var/netflow/*
# ls -lah /var/log/flowd*
# df -h


Thanks,
Franco

Title: Re: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 18, 2017, 10:21:03 am

Hi franco,

Here it is:

Code: [Select]

root@opnsense:~ # ls -lah /var/netflow/*
-rw-r-----  1 root  wheel    96K Nov 18 05:01 /var/netflow/dst_port_000300.sqlite
-rw-r-----  1 root  wheel   684K Nov 18 05:01 /var/netflow/dst_port_003600.sqlite
-rw-r-----  1 root  wheel   1.9M Nov 18 05:01 /var/netflow/dst_port_086400.sqlite
-rw-r-----  1 root  wheel   2.5M Nov 18 05:01 /var/netflow/interface_000030.sqlite
-rw-r-----  1 root  wheel   1.1M Nov 18 05:01 /var/netflow/interface_000300.sqlite
-rw-r-----  1 root  wheel   116K Nov 18 05:01 /var/netflow/interface_003600.sqlite
-rw-r-----  1 root  wheel    16K Nov 18 05:01 /var/netflow/interface_086400.sqlite
-rw-r-----  1 root  wheel    12K Nov 10 20:57 /var/netflow/metadata.sqlite
-rw-r-----  1 root  wheel   160K Nov 18 05:01 /var/netflow/src_addr_000300.sqlite
-rw-r-----  1 root  wheel   364K Nov 18 05:01 /var/netflow/src_addr_003600.sqlite
-rw-r-----  1 root  wheel   384K Nov 18 05:01 /var/netflow/src_addr_086400.sqlite
-rw-r-----  1 root  wheel   7.8M Nov 18 05:01 /var/netflow/src_addr_details_086400.sqlite


Code: [Select]

root@opnsense:~ # ls -lah /var/log/flowd*
-rw-------  1 root  wheel   7.6M Nov 10 20:41 /var/log/flowd.log
-rw-------  1 root  wheel    11M Nov  9 03:26 /var/log/flowd.log.000001


Code: [Select]

root@opnsense:~ # df -h
Filesystem           Size    Used   Avail Capacity  Mounted on
/dev/ufs/OPNsense     27G    1.0G     24G     4%    /
devfs                1.0K    1.0K      0B   100%    /dev
devfs                1.0K    1.0K      0B   100%    /var/dhcpd/dev


Title: Re: flowd_aggregate.py very high cpu usage
Post by: wizziLalev on November 20, 2017, 08:25:54 pm

@franco, any clue?

Title: Re: flowd_aggregate.py very high cpu usage
Post by: franco on November 20, 2017, 10:53:19 pm

Unfortunately no. We have to apply more force, can you reinstall all packages and reboot?

# opnsense-update -fp


Cheers,
Franco