Swap issue completely blocks OPNsense

[bongo]

about 2 weeks ago, i updated from 21.7 to 22.1.
i actually run the latest version, i.e. all updates are installed.
so far, everything worked fine, but this morning, i was no longer able to reach the internet from the local lan.
i was also not able to login to the OPNsense, i.e. the ip address of the OPNsense router did not answer.

when looking at the screen attached to OPNsense (it's running on PC hardware), i've seen lots things that got killed because of a swap issue (i unfortunately did not take a picture).

i tried to do a controlled reboot, but was not able to shut down the machine. so all that helped was to press the reset button.

anyone else having such problems?
any idea how to avoid this?

thanx for your help!

[gleepwurp]

Hi,

Same issue here.  Although system has 12GB of RAM, the swap starts getting filled up around 8h am, and grows so much that opnsense becomes unavailable. traffic does keep on trucking through for a certain time, although unbound stop resolving DNS entries before opnsense become totally unresponsive.  Tries logging in through SSH, and doing the top command doesn't show any processes hogging the ram, even though the System RAM utilisation graph shows Swap full, but interestingly enough shows over 60% Free RAM.

I'm trying to uninstall any addons and disabling services individually to see which one is causing it.

[gleepwurp]

Forgot to says that this happened on 2 different systems, one with 8GB Ram, and currently on my poweredge 1950 with 12GB RAM. So, doesn't seem like a driver/system issues, but a config one.

[HerrDevilmann]

I'm affected by the same phenomenon, I think. Prior to system failure, everything appears normal. CPU, memory, swap, and disk usage all within normal parameters. Four minutes later, without any indication of increased resource usage, the console is flooded with system messages of the form pid xxxx (yyyy), jid 0, uid zzz, was killed: out of swap space. System becomes completely unresponsive, necessitating hard reset.

[Patrick M. Hausen]

Let "top -o res" run in an ssh session until the system crashes again to find, which process is using all the memory.

[ATL]

Do you have /var RAM disk and /tmp RAM disk settings active? If yes, all files below this directories fillup the memory including swap. I had the same issue after switch from 21.7 to 22.1: Over time the memory fills up until the OPNsense doesn't react on anything and had to be hard reseted.

My investigations showed, that many log files were stored und this the memory filled up. I reconfigured System -> Settings -> Logging -> Preserve logs (Days) to 3 and changed the Log level verbosity of Unbound to 1. Now the memory usage keeps below 50% and OPNsense is working without any problem over weeks.