[SOLVED] High load

[Tripple_Delta]

Hi,

After a few days the CPU load of my box goes up even when there's no traffic.
Load average   2.36, 2.26, 2.20

A reboot fixes the problem for a few days.
Looks like a Pyton script is the cause.

Poor hardware?

[bartjsmit]

Hard to say if it is hardware without details ;-)

Open a shell and run top -a when CPU shoots up again

Bart...

[Tripple_Delta]

The hardware is a Vasco AxSguard AG2504.
CPU is a Intel(R) Atom(TM) CPU N450 @ 1.66GHz (2 cores) with 1GB RAM.

top -a shows:
last pid: 88264;  load averages:  2.77,  2.57,  2.35                                                                                            up 7+06:54:05  16:32:26
65 processes:  3 running, 62 sleeping
CPU: 73.2% user,  0.0% nice, 25.6% system,  0.0% interrupt,  1.2% idle
Mem: 26M Active, 684M Inact, 209M Wired, 108M Buf, 35M Free
Swap: 2048M Total, 2048M Free

10723 root          1  21    0    99M 17676K select  1  29.8H   25.57% /usr/local/bin/python2.7 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py

This line fluctuates from 5% up to 70%

[bartjsmit]

I'm seeing a similar result on a single core Celeron D 3GHz with load between 0.7 and 1.2.

You could try disabling netflow if you are seeing dropped packets or high latency on the web interface.

Bart...

[franco]

Hi guys,

Hmm, yes, local netflow (insight) usage can generate load and disk writes, but...

(1) using a netflow reporting frontend will do that, the numbers need to be crunched

(2) cpu idle time is important too. load just means there is X processes scheduled at time Y and if you have a lot of processes they all get to wait for their turn, but maybe they just wake up to realise they need to sleep.

(3) don't forget that a load of 2.3 on a 2-core system is actually 1.15, if it's a hyper-threading env it's actually 0.5075

The http://ark.intel.com/products/42503/Intel-Atom-Processor-N450-512K-Cache-1_66-GHz has 2 cores with HT so it's not even overly loaded.


Cheers,
Franco

[franco]

I just saw the CPUs seem to be saturated which could be because there is a larger amount of traffic being pushed through the system (a few hundred Mbit maybe), which gets then captured locally, and is then read locally. My guess is that the system is doing generally ok, but may be throttling throughput in order to cope with the local reporting?

Traditionally, reporting solutions are extra appliances to avoid that.

[Tripple_Delta]

I forgot about the cores. CPU is actually doing fine indeed.
Is it possible to restart OPNsense without a reboot?
Someting like #service opnsense restart ?

[Tripple_Delta]

Running OPNsense 16.7.8-amd64, I don't see this high load anymore.

Load average    0.38, 0.30, 0.26

[franco]

Hi Tripple_Delta,

Glad to hear!

There is one candidate that might have caused this (fixed a few versions ago):

https://github.com/opnsense/core/issues/1240


Cheers,
Franco