Appliance running hotter after 22.1 update?

Started by Mundan101, January 28, 2022, 02:10:36 PM

Previous topic - Next topic
January 28, 2022, 02:10:36 PM Last Edit: January 28, 2022, 02:12:25 PM by Mundan101
I have a Qotom i7 that I've been running Opnsense on which consistently stayed about 50-55 prior with 21.7.  Now  its between 63-70 Celsius.

No errors, or anything out of the ordinary that I can tell so far.

Using a shell, can you run "top" and see if you have a process that running at 100% ?

On my little APU2 after upgrade, unbound was running at 100% and never dropped to idle.

In Unbound DNS: Blocklist, I removed all of the blocklists and even disabled DNSBL.
When I applied the changes, unbound started and dropped to idle right away.
Then, I enabled DNSBL and added blocklists.
I add several blocklists, apply, and unbound starts, adds the blocklists, and drops to idle relatively quickly.
However, there is one very large (50 MB) blocklist that seems to cause constant 100% load for unbound. I'll investigate further this weekend.

Checked, nothing is running at 100%....

So far no performance lags in the network.

my quotom i7 -    Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (4 cores, 8 threads)
start at 60-65 celsius and then goes down to 34.
Depending on CPU usage it stays between 30 and 50.

Yes i feel an increase as well.
But the throughout seems to be higher so i guessed it came from the suricata multi core update.
i do run suricata on wan. Lot of drop rules, DNSBL unbound and sensei on all the LAN interfaces.
English: Never try, never know!
Deutsch: Unversucht ist Unerfahren!

have a look at https://forum.opnsense.org/index.php?topic=26590.0. It could be that the CPU is running in turbo mode so runs hotter but will ramp down after a while. Or it could be that you need to enable SpeedShift or equivalent in the BIOS and make sure CPU Boot Performance is set to Max Non-Turbo Performance

If your dmesg shows hwpstate_intel as active, you can also tune it t achieve the temps you want via dev.hwpstate_intel.X.epp=[0-100] where 0 = best performance and 100=most energy efficient operation. Take a look at https://www.neelc.org/posts/freebsd-speed-shift-laptop/ and https://www.freebsd.org/cgi/man.cgi?query=hwpstate_intel&apropos=0&sektion=4&manpath=FreeBSD+13-current&arch=default&format=html

So now upon a reboot i noticed that i received the core temp high warning.  The temps do go down after the system starts all services and becomes stable.




I believe you are correct and replied to your post on reddit with my findings. Opnsense is slower if highadaptive is not used because under some cases it uses only 1 core and this causes issues with adaptive governor in powerd. This means you must run the appliance hotter to get performance in the test you described. There is no way around it if opnsense only uses 1 core for certain things.

Findings  show me the upgrade to 22.1 is more resource intensive, in the most simplistic newbie way i could determine.  Things that's worked under 21.7 is now pushing the CPU on 22.1 with my deployment hardware of choice.

I don't think any of my deployments are out of the ordinary however I have IDS/IPS, Zenarmor, Wireguard, OpnVPN, VLAN, telegraph, netdata and ntop running.  Majority of my deployments are Qotom i7-7500U and worked great!

I've tested a few boxes so far with 22.1 and it pushes the single core high in terms of usage.  1 box as already seen performance decrease, removing ntop helped stabilize.

Time for a hardware upgrade it seems....wahooo

February 09, 2022, 10:04:36 AM #8 Last Edit: February 09, 2022, 10:20:57 AM by cephyr
Hi,

I am experiencing the same thing, much more cpu usage than on 21 version.
I've opened top and it seems that ntopng was using much cpu. I disabled it now and see how it goes.


edit:

okay the biggset cpu impact is gone now but there is still a python process spiking cpu again and again

process in top:

24512 root          1  32    0    42M    29M select   3  73:13  36.43% python3.8



root@OPNsense:~ # procstat -f 24512
  PID COMM                FD T V FLAGS    REF  OFFSET PRO NAME
24512 python3.8         text v r r-------   -       - -   /usr/local/bin/python3.8
24512 python3.8          cwd v d r-------   -       - -   /
24512 python3.8         root v d r-------   -       - -   /
24512 python3.8            0 v c rw------   4       0 -   /dev/null
24512 python3.8            1 v c rw------   4       0 -   /dev/null
24512 python3.8            2 v c rw------   4       0 -   /dev/null
24512 python3.8            3 v r -w-----l   1       5 -   /var/run/flowd_aggregate.pid
24512 python3.8            4 v c rw------   4       0 -   /dev/null
24512 python3.8            5 s - rw------   1       0 UDD /var/run/logpriv


I am also running a Qotom i7 appliance

Yes, I can see the similar.
Before 22.1 my CPU was always ~35-38 C with rare jumps up to 42 C.
After upgrade to 22.1, CPU stays at 48-52 C with periodic jumps to 39-41 C. I never saw temperature above 50 C before 22.1. Something definitely keeps freqs (and temperature) at much higher values than before.

Can confirm this too, quite visible jump from average "idle" of 36'c to 40'c just by upgrading. Did an in-place upgrade with no (intentional) simultaneous config changes. Box is custom x86 PC with 4 GHz Intel CPU.


I have noticed high cpu usage on my atom machine as well with the new version.  It's not pinning a core but i see constant spikes to 50 percent total usage.  It's usually python so i wonder why this newest version is doing in python that's so intensive.

Obviously we have notable difference compared to previous version.

Any suggestion what to check?
Any terminal command?
Anything else?

Has anyone benchmarked FreeBSD 12 vs. FreeBSD 13 yet? This could be a significant change, but nobody really asks why and how it impacts performance and system responsiveness.


Cheers,
Franco

There are several possible sources of temperature difference:

  • The base OS may load CPU differently
  • The OS may control CPU differently
  • The new version of OPNsense base framework may produce more CPU load
  • The new version of OPNsense may load CPU more
It could be any of these factors or combinations of them.

For me the throughput and the latency the same as before (the throughput is limited by my internet connection), no difference.

Maybe someone has spare set of equipment to make a clean OS test FreeBSD 12.1 / FreeBSD 13 (or even better: compare HardenedBSD 12.1 with FreeBSD 13). Unfortunately I don't have such ability.

However, I can make some other tests and checks on the live working equipment.
For me it looks like OPNsense 22.1 does not let system to be idle even without notable traffic, while with pervious versions of OPNsense system was almost idle with light traffic.

Could you advise me the best way how to check this assumption? Even negative result would be informative.