Hello,
i am evaluating Sensei for some weeks. The appliance has a disk of 64GB, 20% used. With Sensei + MongoDB 23% is used (when erase all reporting data), max number of days is 1.
Now the interesting part: disk usage grows slowly (3-4 days) to about 50%, then all of a sudden 70-75% and growing very fast to 80%++ (within 1-2 hours), where i overreact and reset reporting data again. After erasing, back to 23% disk used.
My question: what happens, when i would not reset the reporting data in Sensei? Is there an automatic cap of used disk space before the disk would run to 100% full? would anything crash? i had no nerves to test that in the production environment. my testing environment has not enough traffic to get into such scenario.
Thanks in advance
edit:
all newest stable updates installed (opns 21.1.8, sensei enigne 1.9.2, ui 21.7.7, db 1.9.21070514)
issue persist since i installed sensei 8-9 weeks ago
sensei is in passive mode, level warning, deployment size small, rotation 1 day, max days to store reporting data 1 day (not a lot of traffic except the usual attacks from outside)
My installation on an APU2 occupies about 13 GB in total, including toolchain for compiling binaries from source code. YMMV, I don't know how many devices you have on your LAN.
Oh, yes, I have no swap file, I killed the swap partition.
Hi @flames,
Sensei will stop working when detected that a partition is %90 + full. But did you look that which folder is growing fastly? It is not expected behavior for DB data.
You can send your logs to a remote server running elasticsearch to lower the space used on the device running the firewall.
Quote from: sy on July 19, 2021, 09:24:15 PM
Sensei will stop working when detected that a partition is %90 + full. But did you look that which folder is growing fastly? It is not expected behavior for DB data.
thank you. it is indeed mongodb index files (/usr/local/datastore/mongodb/*.wt, a lot of them), i also found something similar in the mongodb community https://www.mongodb.com/community/forums/t/mongodb-disk-space-increases-abruptly-wiredtigerlas-wt/3620
will investigate further.
also i am now a little more relaxed by the 90% disk limit, in case i am not around, when it grows again.
I have another APU2 21.7 with Sensei, it has 9 GB in total.