SSD get's massively written to - cannot find the reason

Started by senseOPN, November 19, 2025, 05:21:12 PM

Previous topic - Next topic
I noticed that the TBW (Tera-Byte Written) of my SSD used for OPNsense shows about 4 TB that got written.

I got the current values yesterday and today again:

root@OPNsense:~ # ls collect_mb_2025-11-18 collect_mb_2025-11-19
-rw-r--r--  1 root wheel 11 Nov 18 23:41 collect_mb_2025-11-18
-rw-r--r--  1 root wheel 11 Nov 19 17:01 collect_mb_2025-11-19

root@OPNsense:~ # diff collect_mb_2025-11-18 collect_mb_2025-11-19
1c1
< 3992034 MB
---
> 4024636 MB


This is about 32 GB in not even 21 hours!

I tried to catch the process that is responsible with iostat, procstat, top (top -b -m io -o write) and zpool iostat.
But I see no process that is doing this.

From the process list, one python3.11 for NetFlow and vnstatd are generating the most writes, but by far not enough for this 32 GB!

I have no idea what to do or check next.

Any idea?

I am using the regular zpool / zfs configuration and have "copies" set to 3, which should tripple write-IOs, but that is not the problem here.

The only suspect I have, is ZFS itself, with it's ARC cache that tries to predict future IO.
dtrace is of course installed, but does not seem to work:

root@OPNsense:~ # dtrace -n 'proc:::exec-success { trace(execname); }'
dtrace: invalid probe specifier proc:::exec-success { trace(execname); }: "/usr/lib/dtrace/psinfo.d", line 37: syntax error near "uid_t"

/var/log and /tmp are configured as RAM disks with 20% of memory each.

Otherwise, I am at version 25.7.7_4-amd64


Netflow. It's by far the generator of the largest amount of any data you can have on OPNsense. Export Netflow data to an external collector like ElastiFlow

But then again let's assume you write 40 GB per day. With a typical TBW of 200 for a smaller quality SSD that means you get 5.000 guaranteed days of operation or over 13 years.
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)

November 19, 2025, 11:02:14 PM #2 Last Edit: November 19, 2025, 11:40:26 PM by senseOPN
Quote from: Patrick M. Hausen on November 19, 2025, 06:18:34 PMNetflow. It's by far the generator of the largest amount of any data you can have on OPNsense. Export Netflow data to an external collector like ElastiFlow

But then again let's assume you write 40 GB per day. With a typical TBW of 200 for a smaller quality SSD that means you get 5.000 guaranteed days of operation or over 13 years.

Many thanks, I will disable NetFlow for now and check if this betters the situation.

While you may be right about the usage, I just don't like the idea that more data get's written to the SSD than I am transferring over the net in the same time.
That is just not right.