CPU spike every 60 seconds from update_tables.py

[wondercow]

Hello--new opnSense user here running 20.1.8_1 as a home router. I'm seeing a significant spike in CPU every 60 seconds from update_tables.py. Is this expected behavior? I've turned basically everything off and have only auto-generated firewall rules (block private addresses/bogons on WAN, no ipv6 through firewall), and I'm still seeing this. Crontab has update_tables running on the minute, every minute--is this how it ought to be? See attached screenshot of cpu spikes from netdata.

crontab (note especially the bottom line):

Code Select Expand

#minute hour    mday    month   wday    command
1       *       *       *       *       (/usr/local/sbin/expiretable -v -t 3600 webConfiguratorlockout) > /dev/null
2       *       *       *       *       (/usr/local/sbin/expiretable -v -t 3600 sshlockout) > /dev/null
3       *       *       *       *       (/usr/local/sbin/expiretable -v -t 3600 virusprot) > /dev/null
5       *       *       *       *       (/usr/local/etc/rc.expireaccounts) > /dev/null
*/4     *       *       *       *       (/usr/local/sbin/ping_hosts.sh) > /dev/null
0       1       *       *       *       (configctl system remote backup) > /dev/null
11      1       *       *       *       (/usr/local/etc/rc.dyndns) > /dev/null
1       3       1       *       *       (configctl filter schedule bogons) > /dev/null
*       *       *       *       *       (/usr/local/bin/flock -n -E 0 -o /tmp/filter_update_tables.lock /usr/local/opnsense/scripts/filter/update_tables.py) > /dev/null

[ermannobonfazi]

Do you have Netflow v9 enabled? I saw this happening and from other comment being related to this. I shifted to v5 (more than enough for OPN) and is really almost disappearing or lot less CPU spike. Hope this help.

[wondercow]

Thanks, I also used to have a second recurring CPU spike related to that too. In the process of troubleshooting this, I've turned off just about every feature on opnSense. From watching top when this happens and checking on the PIDs, I can see that there are two threads of update_tables.py that launch every minute, on the minute, and saturate one CPU core apiece.

Thanks for the tip about v9 though. When I start turning things back on, I'll try out v5. Back when I had netflow turned on, I also saw that saturating one CPU core for about 10 seconds every minute.

[wondercow]

I take it back, it's not two threads of update_tables.py, it was one thread of that and one thread of the et-pro telemetry edition (even though I have intrusion detection currently turned off!). I thought that was only supposed to get a heartbeat every 30 minutes, not every minute.

I have uninstalled the os-etpro-telemetry plugin entirely and now that CPU spike is gone. That seems like a bug to have it using CPU even when IDS is turned off and no token is installed, no?

I also edited cron to make update_tables.py run every 10 minutes instead of every minute, and now my CPU usage chart is nice and flat when the network is at idle.

That said, did I do a bad thing by changing the frequency that update_tables.py runs? I don't really understand what the importance of running that script once per minute is.

[ermannobonfazi]

I check the script and just update the pf tables. Maybe that frequency is useful in scenario where you have HA enabled and want to be sure the 2 system are constantly aligned (just supposing, sorry if wrong).