21.1.9 Upgrade & Insight Aggregator CPU Usage

Started by rman50, July 27, 2021, 03:51:27 PM

Previous topic - Next topic
After upgrading to 21.1.9 from 21.1.8 a few hours ago, the Insight Aggregator python process has been pegging the core it is running on at 100%. I tried a reboot and it didn't help. Anyone else running into this issue or have a suggestion? I am running a Netflow data repair now to see if that helps and will try a Netflow reset after that.

same problem
Supermicro A2SDi-4C-HLN4F
Team Rebellion Member (sidebar / themes: tukan, cicada & vicuna)

Reboot.

It fixed it for me ... or so it seemed before I left for work

Cheers,

July 27, 2021, 04:00:55 PM #3 Last Edit: July 27, 2021, 07:03:53 PM by gpb
I thought I had that problem as well (had it in the past) when I saw high CPU usage, but running htop I noticed it was a different python process "/usr/local/service/configd_ctl.py".  Manually terminated it and all was fine.

Edit: Add screenshot
HP T730/AMD  RX-427BB/8GB/500GB SSD
HP NC365T 4-PORT

Check dmesg. HBSD SEGVGUARD is likely blocking syslog-ng restart as some others have reported. Reboot is an option too.


Cheers,
Franco

I had confirmed it was this process: /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.7). A Netflow reset seems to have resolved the issue for me at the moment.

Alright, Python was updated from 3.7.10 to 3.7.11 but need to look into why that would cause a hang...


Cheers,
Franco

July 27, 2021, 06:32:25 PM #7 Last Edit: July 27, 2021, 06:36:23 PM by Taomyn
Add me to the list of victims, did my upgrade to 21.1.9 about 30min ago then suddenly I could a fan going mad that turned out to be my firewall and the CPU at around 94%, usually sits around 5-10%. Tried just reloading all services, it got worse, so just rebooted and will watch it again.


Let me know if there is any particular info you need from me to diagnose.


Update: Zabbix didn't catch the CPU but it did see a huge spike in RAM usage

Hi,

in my case it was the /usr/local/bin/python3 /usr/local/opnsense/service/configd_ctl.py -e -t 0.5 system event config_changed (python3.7) process. Killing it, solved the issue for now.

KH

I have exactly the same problem on two systems I upgraded today. However, after a reboot both seem to be working fine for now.

Quote from: KHE on July 27, 2021, 06:41:35 PM
Hi,

in my case it was the /usr/local/bin/python3 /usr/local/opnsense/service/configd_ctl.py -e -t 0.5 system event config_changed (python3.7) process. Killing it, solved the issue for now.

KH

Same here.

Quote from: rman50 on July 27, 2021, 04:11:15 PM
I had confirmed it was this process: /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.7). A Netflow reset seems to have resolved the issue for me at the moment.

this also works for me. thx :)
Supermicro A2SDi-4C-HLN4F
Team Rebellion Member (sidebar / themes: tukan, cicada & vicuna)

I had to kill -9 the python process. Hopefully thats alright. Restarting/resetting netflow didnt help for me.

+add me to the list...

syslog 1 core @100%
configd 1core @100%

I upgraded two systems and both had the same issue. Killing the offending python3.7 process fixed the issue on both systems.