Help tracking down Out of Swap issue

[Szeraax]

Hi all, love my OpnSense. Except lately, its been running out of swap space and crashing. I haven't been able to find a clean and easy way to view all the ram usage per process to track down the cause of the issue. Here's what my system memory overview for the last 20 hours looks like: https://i.imgur.com/N0vb3zP.png (And 60 hours: https://i.imgur.com/6Oa1gbc.png)

I have 4GB of ram installed on this machine. It is running a AMD Ryzen V1000 in a Hyper-V guest VM. (https://www.newegg.com/asrock-4x4-box-r1000v/p/N82E16856158066). Two vNICs installed in the VM.

Looking at `top`, I can't find any way to sort by most memory used (people say to use shift m, i, etc. but none of that works on my shell at least). But I can see that I have multiple nginx processes using 100M+ of ram. Seen here: https://i.imgur.com/0HLgKic.png

Looking at another way to view ram per process, I see that ps lists the %MEM stat and the VSZ stat. If I sort by either of those methods, I still don't know how to track this down to what subsystem of OpnSense is leaking ram and causing my router to require multiple reboots every week for my regular home (<20 devices) network. Shown here: https://i.imgur.com/8oDylu8.png

I'm leaning towards this being an issue with nginx just because I have like 20 HTTP servers (and 20 locations and 20 upstreams and 20 upstream servers) to services in my network and nginx is one of my top memory users.

Does anyone have any tips or tricks for me to figure out exactly what the cause is? If nginx is the root cause, are there any good ways to track down what's up? Looking at the nginx global log, there are a few errors about ssl_early_data not supported and getting ignored. Or a duplicate server name getting ignored. Certainly nothing big that suggests nginx sucking up all my ram.

Thanks all!

[Patrick M. Hausen]

Looking at `top`, I can't find any way to sort by most memory used (people say to use shift m, i, etc. but none of that works on my shell at least). But I can see that I have multiple nginx processes using 100M+ of ram. Seen here: https://i.imgur.com/0HLgKic.png

FreeBSD top is different but also more powerful than the one that comes with most Linux distros. Might be the reason why everyone installs htop on Linux ...

While top is running try hitting '?' for builtin help. To sort for resident memory size try 'o' followed by 'res' and the enter key. For swap used try 'o' followed by 'swap' and the enter key. Try invoking `top -w` before sorting for swap to see even more relevant data.

HTH,
Patrick