Re: Random Frequent CPU Spikes and Page Faults [Almost Resolved]

[dpsguard]

Thanks @zz00mm for your advice. However in my case, system is not busy as all, it is idle with essentially no traffic. And I get that page faults is not bad, but there is no use of any swap either and overall RAM use is a fraction of what is available. My more concern now is on the constant interruption by python.

[opnfwb]

I finally got around to sampling the old OPNsense 19.1.4 image I had available. This is quite old, based on FreeBSD 11 so I don't think this is a relevant comparison at this point but I'm posting the results anyway. It does show noticeably lower faults than the current versions and it has the same hardware config and the same plugins installed (vnStat and vmware tools).

This vmstat sample was taken with the system effectively idle, just passing minimal gateway monitor ping traffic.

OPNsense 19.1.4

Code: [Select]

procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr da0 cd0   in    sy    cs us sy id
0 0 0 2.0T  1.6G     7   0   0   0     0   42   0   0    7   418   151  0  0 100
1 0 0 2.0T  1.6G     0   0   0   0     0   43   0   0    6   233   130  0  0 100
0 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    3   194   117  0  0 100
0 0 0 2.0T  1.6G     3   0   0   0     0   42   0   0    9   235   149  0  0 100
0 0 0 2.0T  1.6G     1   0   0   0     0   42   0   0    3   223   121  0  0 100
4 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    7   206   127  0  0 99
1 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    3   222   125  0  0 100
0 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    6   209   134  0  0 100
0 0 0 2.0T  1.6G     4   0   0   0     8   42   5   0   12   256   165  0  0 100
2 0 0 2.0T  1.6G     1   0   0   0     0   42   0   0    8   227   141  0  0 100
0 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    4   248   122  0  0 100
0 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    6   218   132  0  0 100
2 0 0 2.0T  1.6G     0   0   0   0     0   84   0   0    5   209   123  0  1 99
0 0 0 2.0T  1.6G     2   0   0   0     0   42   0   0    3   202   121  0  0 100
0 0 0 2.0T  1.6G     0   0   0   0     0   42   1   0    7   237   148  0  0 99
0 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    4   204   127  0  0 100
0 0 0 2.0T  1.6G 11701   0   0  11 11041   42   0   0    5  8359   485  2  3 95
0 0 0 2.0T  1.6G 23435   0   0  16 23041   44   0   0    7  7984   548  3  5 92
0 0 0 2.0T  1.6G     2   0   0   0     0   42   1   0    5   247   137  0  0 100
1 0 0 2.0T  1.6G     0   0   0   2     0   42   2   0    5   211   132  0  0 100
0 0 0 2.0T  1.6G     0   0   0   0     0   42   0   0    6   224   136  0  0 100

[dpsguard]

Thanks again @opnwb for your continued interest and help with my situation. I can see same type pf page faults in your 19.1.4 tests as I saw with FreeBSD 14 based pfsense latest version.

I am also dealing with another issue described here at

https://forum.opnsense.org/index.php?topic=35288.0

I may also try installing version 19.1.4 to see if it resolves both of my issues. My needs are simple, just a NAT router with Captive portal for Guests to accept terms and then allowed to go to Internet.

[opnfwb]

I would highly recommend avoiding the older installs of any firewall distro. They aren't security maintained and will only become more vulnerable over time. In this case we're talking about falling back to something two major OS revisions behind with no future support.

I just did this as an interesting baseline to see if I saw the same vmstat results (I don't) compared to newer verions. Beyond that, I wouldn't seriously consider still running something this old and I wouldn't recommend it to anyone either. Just my 2c on it but there you have it.

[dpsguard]

Thanks @opnfwb for your advice. I agree with what you said.

[dpsguard]

@opnfwb can you please also review my post below in case you have any experience with similar issues? Thanks

https://forum.opnsense.org/index.php?topic=35375.0

[libertasfox]

Hey All,

I am jumping on the proc at 100% bandwagon with this last upgrade as well.  I was having no issues with the FW until I upgraded to 23.7.  The latency of UI is painful and sometimes fails to even load.  I'm not having any bandwidth as of now but the proc and core temps are running consistently high.  I'm also running Zenarmor and with their UI upgrade I wonder if this is having an effect??  Anyway, just wanted to add to the list of end users who are having this issue and hope the folks at OPNsense are actively looking into this.

[dpsguard]

In my case, I seem to have lowered the CPU spikes to seemingly acceptable levels. I am running latest version and I never had 100% CPU issues, rather mostly CPU was very low and then sudden spikes that will go sometimes to 70%.

What I found was that under Firewall/Settings/Advanced, Firewall Optimization was set to aggressive. I changed it to normal. Further I removed all the widgets on dashboard and then also if I don't use the GUI, then repeating the tests that I was doing, I don't see the ping drops from time to time (which I attributed to CPU peaks).

Only issue that I have (or my misunderstanding) is around Python Captive Portal background / housekeeping script showing as 100% IOPS every couple of seconds.