configd_ctl.py eating memory

[GaardenZwerch]

Hi,

out of my >50 opnsense boxes there is one that regularly runs out of memory over the course of around two weeks, to the point where they have to powercycle it on site.

Yesterday, I got a chance to look after it before it was to late.

top shows me its a python process:

Code Select Expand

last pid: 22346;  load averages:  3.76,  2.84,  1.89             up 8+10:04:59  17:52:52
53 processes:  4 running, 49 sleeping
CPU: 30.1% user,  0.0% nice, 12.7% system,  0.2% interrupt, 56.9% idle
Mem: 5423M Active, 62M Inact, 776M Laundry, 1226M Wired, 697M Buf, 220M Free
Swap: 8192M Total, 3754M Used, 4438M Free, 45% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
65714 root          1 103    0  5667M  5136M CPU7     7 100:34  97.62% python3.7
38963 root         17  41    0  1122M    14M sigwai   7   0:02   0.00% charon
84282 root          1  20    0  1050M  4360K select   3   0:27   0.00% sshd


and here are the python processes:

Code Select Expand

root@lesc:~ # ps aux|grep python
root   65714  99.0 64.4 5803116 5258816  -  R     9Dec20    99:41.55 /usr/local/bin/python3 /usr/local/opnsense/service/configd_ctl.py -e -t 0.5 system event config_chang
root    7298   0.0  0.1   20540   10716  -  S    17:47       0:00.06 /usr/local/bin/python3 /usr/local/opnsense/service/configd_ctl.py -e -t 0.5 system event config_chang
root   13583   0.0  0.3   58004   21716  -  I     9Dec20     0:12.39 /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.7)
root   20151   0.0  0.0   31876       0  -  IWs  -           0:00.00 /usr/local/bin/python3 /usr/local/opnsense/service/configd.py (python3.7)
root   27623   0.0  0.1   21076   11228  -  S    17:47       0:00.07 /usr/local/bin/python3 /usr/local/opnsense/scripts/syslog/lockout_handler (python3.7)
root   93887   0.0  0.2   37548   16812  -  Ss    9Dec20   825:04.49 /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.7)
root   65791   0.0  0.0 1067288    2932  0  S+   17:51       0:00.00 grep python


So, I'm wondering what "configd_ctl.py -e -t 0.5 system event config_chang" does, why there's two of them, and how I can find out what's going wrong. At the time I got to take a look, the systemlog was full off 'cannot allocate' entries.

The system has 8GB Ram + 8GB Swap. No suricata, no flowd. It is running 20.7.6 but it has shown this with earlier 20.7.x releases.

Thanks for any hints,

[mindfuse]

I share the same problem on my small APU1D. /usr/local/opnsense/service/configd_ctl.py Eating up 65-100% cpu flat out.

Type   opnsense   
Version   21.1.9_1   
Architecture   amd64   
Flavour   OpenSSL   
Commit   e10896164   
Mirror   http://mirror.fra10.de.leaseweb.net/opnsense/FreeBSD:12:amd64/21.1   
Repositories   OPNsense

I need to reboot it too. The only thing I can think of might have triggered it: I updated the DNS list in System > Settings > General a few days ago. It might've since then been acting up?

Any pointers?

Cheers
M