nightly shutdown of Zenarmor due to (false-positive) disk usage warning?

Started by granute, March 21, 2025, 03:48:36 PM

Previous topic - Next topic

This is as been ongoing issue for several months now and I'm not sure what to make of it. My Zenarmor Home shuts down every night due to what appears to be a false-positive warning about disk usage on the OPNsense box. In my monitoring I see no indication that any of the volumes on the firewall are going above 50%.

Other than this being a bit of a hassle -- restart every morning -- it also means that my family web filtering is not effective unless I stay on top of things daily.

Anyone else seeing this?

Previous update was in early February but I just updated as I submitted this post. If I don't say otherwise, assume the problem persists even with today's updates applied.

# OPNsense details:

Type opnsense
Version 25.1.3
Architecture amd64
Commit 6aa1d97b1
Mirror https://pkg.opnsense.org/FreeBSD:14:amd64/25.1
Repositories OPNsense (Priority: 11), SunnyValley (Priority: 7), mimugmail (Priority: 5)
Updated on Fri Mar 21 08:39:02 MDT 2025
Checked on N/A

# Zenarmor details

os-sensei (installed) 1.18.6 207MiB 3 SunnyValley Enterprise Security Extensions for OPNsense (ZENARMOR)
os-sensei-updater (installed) 1.17 4.03KiB 3 SunnyValley OPNsense ZENARMOR Plugin Updater
os-sunnyvalley (installed) 1.4_3 2.44KiB 3 OPNsense Vendor Repository for Zenarmor (a.k.a Sensei, Next Generation Firewall Extensions)
os-sensei-agent 1.18.6 115MiB 3 SunnyValley ZENARMOR (Sensei) Connectivity Agent for Cloud Central Management


thx

Hi,

Zenarmor stops the engine service by controlling swap and disc usage. You need to see details in notifications why it has been stopped. Did you check the notifications?



Quote from: granute on March 21, 2025, 03:48:36 PMThis is as been ongoing issue for several months now and I'm not sure what to make of it. My Zenarmor Home shuts down every night due to what appears to be a false-positive warning about disk usage on the OPNsense box. In my monitoring I see no indication that any of the volumes on the firewall are going above 50%.

Hi, I can confirm, I have the same problem.
Versions of the Zenarmor system and packages are the most relevant at the moment.
Why not make automatic wiping of old data, instead of stopping the engine?

Do you use a RAM disk?
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)

Hi,

Zenarmor retires both log and reporting data. However, the growing data is not an old data to retire; rather, it stops the engine service to prevent a system outage.

Sorry, for some reason I'm not getting notification emails from the forum about watched threads. Just happened to login for another reason and see this...

Re RAM disk, sorry I'm not a BSD person and so I'm not totally sure how to check about this and I'm not finding docs on it either. I don't **think** I have RAM disks except perhaps for this:

    tmpfs                        102400      252   102148     0%    /usr/local/zenarmor/run/tracefs

In the web console under Settings/System/Misc none of the RAM disk options are enabled. I did just enable swap there however.

Hi,

This is a temporary partition used to store trace logs for device updates and other activities. To identify the largest folder, navigate to the `/usr/local/zenarmor/run/tracefs/{interface_name}` directory and use the command `du -sh *`. Once identified, you can remove the folder if necessary. Zenarmor retains this data for troubleshooting purposes.

Alternatively, you can increase the partition size by going to Settings > Reporting & Data > Database and adjusting the Tracefs Partition Size. It is recommended to set it to 150 MB to adequately store trace data.



Quote from: sy on April 13, 2025, 10:01:17 PMHi,

This is a temporary partition used to store trace logs for device updates and other activities. To identify the largest folder, navigate to the `/usr/local/zenarmor/run/tracefs/{interface_name}` directory and use the command `du -sh *`. Once identified, you can remove the folder if necessary. Zenarmor retains this data for troubleshooting purposes.

Alternatively, you can increase the partition size by going to Settings > Reporting & Data > Database and adjusting the Tracefs Partition Size. It is recommended to set it to 150 MB to adequately store trace data.


Confirming... I bumped up default values for Memory Disk Size and Tracefs Partition Size significantly and ZenArmor was still running as of this morning. So that's a good sign.

I didn't find much in the docs about doing this for small installations. Some somewhat related material for enterprise deployments.

Thanks for your help, @sy.

Hi @granute,

Thanks for the feedback. I'm going to forward your feedback to the documentation team.


Best regards

Unfortunately I'm back to nightly shutdowns.

I'm looking at what I should install on OPNSense and in my network for log and performance monitoring. Stand by...

Hi,

This is not expected, Can you share a report by following the instructions in the below lin

https://zenarmor.com/docs/support/reporting-bug