Very high memory usage

[nrw]

Appeared after updating to version 23.7.4. Traffic is not high, on average 10-30 megabits/s.

OPNsense 23.7.4-amd64
FreeBSD 13.2-RELEASE-p3
OpenSSL 1.1.1v 1 Aug 2023

Intel(R) Celeron(R) N5105 @ 2.00GHz (4 cores, 4 threads)

Memory usage:
89 % ( 14506/16191 MB )

SYSTEM: DIAGNOSTICS: ACTIVITY

Code: [Select]

247 root 20 0 343M 142M wait 1 0:00 0.58% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 3 0:00 0.30% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 20 0 343M 142M wait 2 0:00 0.26% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 0 0:00 0.24% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 20 0 343M 142M wait 0 0:00 0.13% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 2 0:00 0.13% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 2 0:00 0.12% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 0 0:00 0.08% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 3 0:00 0.07% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M accept 0 0:44 0.03% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}

Services:

Code: [Select]

configd System Configuration Daemon
  cron Cron
  dhcpd DHCPv4 Server
  dpinger Gateway monitor watcher
  dpinger Gateway monitor (MTS_DHCP)
  dpinger Gateway monitor (ROST_DHCP)
  login Users and Groups
  ntpd Network Time Daemon
  pf Packet Filter
  routing System routing
  shadowsockslocal shadowsockslocal daemon
  sysctl System tunables
  syslog-ng Syslog-ng Daemon
  unbound Unbound DNS
  webgui Web GUI


[_Alchemist_]

If you have OPNsense installed on ZFS, this is probably the ARC cache, which uses all free/unused RAM for caching.

In your case, however, it seems that there are a lot of Python processes eating up your available RAM.
Have you tried rebooting your OPNsense firewall?

[nrw]

I used the default installation options. If I'm not mistaken, in this case ufs is selected. After a reboot, 540MB of memory is used and is not growing yet.

[newsense]

Shadowsocks is not in the default install, and may have a memory leak. What happens when you restart the process ?

[nrw]

Shadowsocks is not in the default install, and may have a memory leak. What happens when you restart the process ?

After the restart, the problem has not yet recurred, so I can’t check. But do you really think that shadowsocks can create a bunch of python processes?

[newsense]

May very well be.

I'm not running shadowsocks anywhere and I don't have this issue with anything else running on my FWs

[nrw]

shadowsocks-libev is written in C

[axsdenied]

Appeared after updating to version 23.7.4. Traffic is not high, on average 10-30 megabits/s.

OPNsense 23.7.4-amd64
FreeBSD 13.2-RELEASE-p3
OpenSSL 1.1.1v 1 Aug 2023

Intel(R) Celeron(R) N5105 @ 2.00GHz (4 cores, 4 threads)

Memory usage:
89 % ( 14506/16191 MB )

SYSTEM: DIAGNOSTICS: ACTIVITY

Code: [Select]

247 root 20 0 343M 142M wait 1 0:00 0.58% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 3 0:00 0.30% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 20 0 343M 142M wait 2 0:00 0.26% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 0 0:00 0.24% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 20 0 343M 142M wait 0 0:00 0.13% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 2 0:00 0.13% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 2 0:00 0.12% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 0 0:00 0.08% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M wait 3 0:00 0.07% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}
247 root 52 0 343M 142M accept 0 0:44 0.03% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.9){python3.9}

Services:

Code: [Select]

configd System Configuration Daemon
  cron Cron
  dhcpd DHCPv4 Server
  dpinger Gateway monitor watcher
  dpinger Gateway monitor (MTS_DHCP)
  dpinger Gateway monitor (ROST_DHCP)
  login Users and Groups
  ntpd Network Time Daemon
  pf Packet Filter
  routing System routing
  shadowsockslocal shadowsockslocal daemon
  sysctl System tunables
  syslog-ng Syslog-ng Daemon
  unbound Unbound DNS
  webgui Web GUI


Your diagnostic list looks cut off and it also doesn't by default, sort by memory usage.  You can sort that though.  i.e. unbound is taking most of my memory usage.

[nrw]

Your diagnostic list looks cut off and it also doesn't by default, sort by memory usage.  You can sort that though.  i.e. unbound is taking most of my memory usage.

I sorted by the RES column, i.e. by memory usage.