Resolved: Memory Usage increases up to a point where opnsense stops working

[ThoSt]

I have just upgraded to 25.1.10 last night, and immediately after rebooting, the memory usage keeps increasing.
Free memory declines until the system is unusable, and I need to hard reboot the box.

I am seeing lots of the following in System-Diagnostics-Activity:

Code Select Expand

331 root 20 0 220M 130M wait 3 0:00 0.44% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 20 0 220M 130M wait 0 0:00 0.40% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 20 0 220M 130M wait 0 0:00 0.27% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 68 0 220M 130M accept 2 0:01 0.25% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 20 0 220M 130M wait 2 0:00 0.21% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 68 0 220M 130M wait 2 0:00 0.20% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 20 0 220M 130M wait 0 0:00 0.19% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
331 root 68 0 220M 130M wait 3 0:00 0.13% /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11){python3.11}
I am not sure how to get a handle on this.

I am running the following services, this has been the case for a long time. All updates are applied:

acme   ACME client
configd   System Configuration Daemon
cron   Cron
ddclient   ddclient
dhcpd   DHCPv4 Server
login   Users and Groups
monit   Monit System Monitoring
nginx   Reverse Proxy and Web Server
ntpd   Network Time Daemon
openssh   Secure Shell Daemon
openvpn   OpenVPN server MyServer
pf   Packet Filter
routing   System routing
sysctl   System tunables
syslog-ng   Syslog-ng Daemon
telegraf   telegraf agent
unbound   Unbound DNS
webgui   Web GUI

[ThoSt]

FYI: Did a full reinstall from USB Live, imported my config. same issue.

[ThoSt]

Looks like disabling the WAN_DHCP6 gateway did the trick. Not sure why this was enabled (could this be enabled by default and not have caused issues in prev. builds?).

https://forum.opnsense.org/index.php?topic=46184.msg231245#msg231245

[XabiX]

Hello, it s my 1st time that I see my VM 25.11.1 rebooting within my Proxmox. I have not yet been able to understand why (no crash core dump).

I have disabeled IPv6 GW for now but I still see my mem 90% used in Proxmox which is OK but can"t seem to understand why it crashed.

Code Select Expand

root@OPNsense:~ # top -o res
last pid: 18374;  load averages:  0.16,  0.14,  0.15                                                                                                                                            up 0+00:32:05  16:18:44
83 processes:  1 running, 82 sleeping
CPU:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Mem: 303M Active, 425M Inact, 1448M Wired, 9687M Free
ARC: 613M Total, 121M MFU, 424M MRU, 15M Anon, 4797K Header, 47M Other
     489M Compressed, 1400M Uncompressed, 2.86:1 Ratio

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
62477 root         15  36    0  1401M   116M kqread   3   0:04   0.00% crowdsec
90642 unbound       4  20    0   120M    64M kqread   2   0:00   0.00% unbound
69639 root         10  20    0  1212M    57M kqread   3   0:01   0.00% crowdsec-firewall-b
98397 root          1  20    0    79M    52M nanslp   1   0:05   0.00% php
  377 root          1  68    0    88M    46M accept   0   0:02   0.00% python3.11
22958 root          1  20    0    66M    40M accept   0   0:00   0.00% php-cgi
83348 root          1  23    0    51M    40M nanslp   1   2:25   0.00% python3.11
28331 root          1  20    0    64M    39M accept   0   0:00   0.00% php-cgi
28026 root          1  20    0    64M    39M accept   2   0:00   0.00% php-cgi
25618 root          1  20    0    68M    36M accept   2   0:00   0.00% php-cgi
22629 root          1  20    0    64M    36M accept   2   0:00   0.00% php-cgi
24001 root          1  20    0    62M    34M accept   0   0:00   0.00% php-cgi
25014 root          1  20    0    62M    34M accept   3   0:00   0.00% php-cgi
26897 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
26019 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
26714 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
26207 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
19937 root          1  68    0    56M    28M wait     0   0:00   0.00% php-cgi
25999 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
25255 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
25367 root          1  68    0    56M    28M accept   1   0:00   0.00% php-cgi
21145 root          1  68    0    56M    28M wait     3   0:00   0.00% php-cgi
23418 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
23220 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
24674 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
20534 root          1  68    0    56M    28M wait     1   0:00   0.00% php-cgi
21377 root          1  68    0    56M    28M wait     3   0:00   0.00% php-cgi
30005 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
29462 root          1  68    0    56M    28M accept   2   0:00   0.00% php-cgi
28931 root          1  68    0    56M    28M accept   0   0:00   0.00% php-cgi
62981 nobody        6  20    0  1233M    21M kqread   0   0:01   0.00% node_exporter
93059 root          1  20    0    32M    18M nanslp   1   0:01   0.00% python3.11
  375 root          1  68    0    27M    15M wait     0   0:00   0.00% python3.11
16232 root          1  20    0    28M    15M select   2   0:00   0.00% python3.11
17419 root          3  20    0    45M    15M kqread   0   0:00   0.00% syslog-ng
15648 root          1  20    0    27M    14M select   2   0:00   0.00% python3.11
71043 redis         4  20    0    37M    12M kqread   1   0:01   0.00% redis-server
36847 dhcpd         1  20    0    28M    10M select   2   0:00   0.00% dhcpd
19715 root          1  20    0    23M    10M kqread   0   0:00   0.00% lighttpd
17134 root          1  68    0    24M    10M wait     0   0:00   0.00% syslog-ng
54091 root          1  20    0    20M  9060K select   2   0:00   0.00% sshd-session
16346 root          1  20    0    20M  9052K select   0   0:00   4.96% sshd-session
77473 root          2  20    0    21M  8756K nanslp   1   0:00   0.00% monit
53631 root          1  37    0    20M  8748K select   2   0:00   0.00% sshd-session
Is there any idea?

[meyergru]

Your problem has nothing to do with the OP's problem. Please open new a new thread in such cases.

Obviously, you have allocated 10 GByte of virtual memory, of which most is free. Memory usage of FreeBSD within a Proxmox VM will always be shown as ~90% in Proxmox, regardless of how much is really in use within the VM.

If your VM stops working, then it is because you overcommitted the memory and then, the OOM reaper will stop VM processes as needed. You should be fine with much less allocated memory for your OpnSense VM. 4 GByte should usually be enough, so your overall memory consumption will be 6 Gbytes less and if that can be met, there will be no more process killing.