23.7 CPU / RAM 100% crashing - Zenarmor?

Started by ThyOnlySandman, August 12, 2023, 03:53:40 AM

Previous topic - Next topic
Quote from: rudiservo on August 16, 2023, 12:08:38 PM
Guys, for the sake of trying to figure what is going on, what type of nics are you guys running this.

I am suspecting an issue with zenarmor talking with netmap, I have ZA only on one nic (realtek) and suricata on the wan.
The only time ZA does not go to 100% is if all my networks stop talking, not just the one it is attach to.

ESXI VMXNET3 NICs.  Native netmap.  Zenarmor LAN.  Suricata WAN.
However it does not appear related to traffic inspect.  Issue happens with both Zenarmor + Elasticsearch off.  Even happens on new lab VM without completing the post install setup of Zenarmor.  (Prior to pick DB type, Choose protected interface, etc.)
There will be multiple processes of -  usr/local/bin/php /usr/local/opnsense/mvc/app/library/OPNsense/Zenarmor/CLI.php aliases

I ran handful of tests disabling all addons except unbound to see if I could identify a conflict with another addon.  Just ZA being installed.  Not even running.

Quote from: fatbob01 on August 14, 2023, 09:33:46 PM
I have the same problem on a n100 mini pc. Funny enough, i have the exact same configuration on a esxi vm, no issues.  Had to uninstall zenarmor only on mini pc.  Both 8GB ram.  Let me know what zenarmor has to say.

Thanks!

Interesting and strange.  I shared more details with Sunny Valley but have not gotten a response.

I was able to fix it, I had to reset to factory defaults.

It's in the uninstall tab.

Nonetheless there is another issue, even if you haverouted native netmap, ZA will use emulated netmap.
It's in the Opnsense general logs (debug)

I switch to mongodb as database instead of Elasticsearch since mimugmail repo also uses elasticsearch. Would it be the cause of out of swap page?

Make the tunable dev.netmap.buf_num , set it to 70000 and reboot. YMMV.
OPNsense HW:

Minisforum Venus series UN100C, 16 GB RAM, 512 GB SSD
T-bao N9N Pro, 16 GB RAM, 512 GB SSD

Quote from: fatbob01 on August 14, 2023, 09:33:46 PM
I have the same problem on a n100 mini pc. Funny enough, i have the exact same configuration on a esxi vm, no issues.  Had to uninstall zenarmor only on mini pc.  Both 8GB ram.  Let me know what zenarmor has to say.

Thanks!

I received a response from Sunny Valley today.

"We have a fix for your issue in 1.14.3"

installed 1.14.3 (Existing ZA config still).  Installed ElasticSearch + set native Netmap Protect LAN + OPENVPN INTs.

Didn't reboot, 7 hours uptime since install.  So far so good.
12 vCPU ~5-25%
Memory usage = 52 % ( 6384/12250 MB )
SWAP usage = 8 % ( 700/8192 MB )