High CPU Usage since upgrade from 19.1.10 to 19.7

[cguilford]

Has anyone else noticed almost double high cpu usage compared from 19.1 to 19.7?  As a general rule, my CPU would max about 50% most of the time.. I RARELY saw it peek above that mark, I upgraded to 19.7 this morning and I'm peaking 70-80% frequently.   I've even spiked as high as 91%. 

[pyrodex]

I am seeing a change as well.

[payback007]

I also see the same issue, increased work load and also in combination with a very slow WebUI... any ideas?

It also seems traffic is abnormally high too.

[cguilford]

Yeah I noticed that the webgui was extremely laggy.

[Steven]

What does System: Diagnostics: Activity or top command say are highest CPU usage processes?

[cguilford]

This is what it says currently

[cguilford]

   
PID
STATE
WCPU
COMMAND
   11   RUN   61.08%   [idle{idle: cpu3}]
   21721   CPU3   52.69%   /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.7)
   13223   wait   8.40%   /usr/local/bin/python3 /usr/local/opnsense/scripts/systemhealth/activity.py json (python3.7)
   53112   piperd   2.98%   /usr/local/bin/python3 /usr/local/opnsense/scripts/filter/update_tables.py (python3.7)

[nivek1612]

yes I'm seeing higher CPI than normal

usually no more than 20% peek but now upto 35% (i7 chip)

[Steven]

netflow is using way more CPU cycles than what I would consider normal. In last release it was using python 2.7, I wonder if new python 3.0 usage is causing issue.

For example here is my System Diagnostic Activity while still on 19.1.10:

Code Select Expand

11 root 155 ki31 0 64K CPU0 0 835.0H 92.19% [idle{idle: cpu0}]
88347 root 25 0 1919M 325M select 1 135:00 8.89% /usr/local/bin/suricata -D --netmap --pidfile /var/run/suricata.pid -c /usr/local/etc/suricata/suricata.yaml{W#01-igb1}
27807 root 25 0 39M 32M select 1 73:28 6.49% /usr/local/bin/python2.7 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py
18 root -16 - 0 16K - 3 107:38 2.10% [rand_harvestq]

As workaround to reduce CPU usage you could try to turn off NetFlow. I think you turn off netflow by going to Reporting: Netflow and clear all fields and uncheck capture local.

Or

Maybe try to "Reset Netflow Data" or "repair Netflow Data" at Reporting: Settings.

[cguilford]

I've already done the reset option thinking it will help resolve the problem.  Sadly it did not. I could try turning it off, but that has is open longer-term problem with no data capture.

[cguilford]

So I've turned off netflow completely by wiping out all settings under Netflow, reinstalled the package, ran  "Reset Netflow Data" or "repair Netflow Data" as well as turned off "Round-Robin-Database" the Webgui seems alot more "snappy" now I'm waiting on it to run to see how my processor load looks.


----######Update
So it was running great never spiked over 51% CPU usage with netflow completely wiped out and not running..  I've turned it back on with everything wiped clean including the package reinstall and currently it's running at attached...

[Steven]

That seems much more normal now. Probably monitor it a few more times and see if it spikes again.

[cguilford]

After about 15 hrs of Running it is spiking again it seems...

   
22876   root   52   0   25M   22M   select   1   215:23   35.25%   /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.7)

[gpb]

After about 15 hrs of Running it is spiking again it seems...

Same results here.  It didn't take 15 hrs, about one hour.

[cguilford]

I was going to open an issue on github but seems someone did earlier this morning.
https://github.com/opnsense/core/issues/3587