SDD fast wear ?

[@lex]

Hi,

I just made a quick check on the SMART info of my SSD on which OPNsense runs.

I have a TBW of 430 TB, that is around 100 TBW per year running.

This is enormous and I wonder what setup is recommended to avoid burning SSD Drives within a few years.  It's a small setup with Zenarmor, ntopng, and HAProxy on top of the usual services (DNS, NTP, DHCP).  Nothing fancy and a few dozen different local IPs.

I have to replace my SSD ASAP, and I will put a spare that I have here, but if I keep a similar setup, it will be the same story.

Should I add a mechanical drive and map some of the logging to it ?

What are the best practices ?

[meyergru]

There are already lots of threads about this, use the search, Luke. There are certain things that cause massive writes to disk, Netflow, RRD and excessive logging being in the top list in that order. You can disable each of those and also put /tmp and / or /var/log into a ram disk.

[Lebowski89]

Smells like ZFS. I wouldn't run ZFS-related things on any low-write consumer SSD.

[chrcoluk]

ZFS on opnsense already defaults to 90 sec async flush interval to help wear, I agree with the others, use a ram disk when using SSD storage.

[Taunt9930]

Are there any guides on how to do so / best practice? I had a quick scan of the Docs and couldn't find anything about moving items to a Ram disk. Thanks.

[meyergru]

It is literally in the docs.

[chrcoluk]

In the UI settings.  Read the docs as suggested above.

[JamesFrisch]

Smells like ZFS. I wouldn't run ZFS-related things on any low-write consumer SSD.

"Just turn off ZFS" is the new "Just turn off IPv6". It is mostly misguided.

If you don't have a misconfigures pool that suffers from write amplification (which mostly applies to RAIDZ, which nobody should use for OPNsense anyway), ZFS has double the writes. But that is only for sync writes, not for asynchronous writes.

Even cheap, modern consumer SSDs (that aren't QLC) offer a very high TBW endurance that you will never ever be able to reach with OPNsense defaults.

As a reference, even a cheap 970 EVO 500GB offers 300TBW. After 2,5 years, I got 18TB on my OPNsense.
If the workload keeps the same, I am looking at roughly 40y before I reach the TBW.

[meyergru]

While I second that ZFS is preferable in principle, there are two caveats:

1. The ZFS defaults were different with older versions of OpnSense, such that writes occured more often.
2. There are applications that - if not moved to RAM disk - eat through SSDs quite visibly. Among those are RRD and Netflow.

With my first OpnSense installation (DEC750 on 23.x) and no RAM disk, I consumed half of my enterprise-grade's lifetime in one year.

[Greg_E]

OK, drives are cheap in the sizes we need, so not really something I'm going to worry about. And if you have a mirror boot drive, then you should be able to swap in another drive without having to load the system from bare. That said, loading from a fresh USB might not be a bad idea once in a while, the config file holds most of what is needed for this so normally not a huge issue.

Back to the mirrored boot drive... Can we add a third new disk to the mirror, then remove one of the other disks, add new disk again, then finally remove the last old disk? Truenas apparently can do this, as long as the drives are of identical size. I'd suggest having spares on the shelf for this, but again, small drives are cheap.

[Patrick M. Hausen]

Back to the mirrored boot drive... Can we add a third new disk to the mirror, then remove one of the other disks, add new disk again, then finally remove the last old disk? Truenas apparently can do this, as long as the drives are of identical size.

Any system with a ZFS mirror can do this as long as you have the connectors for all three drives to be online simultaneously. So the answer for OPNsense is "yes, of course".

[CJ]

While I second that ZFS is preferable in principle, there are two caveats:

1. The ZFS defaults were different with older versions of OpnSense, such that writes occured more often.
2. There are applications that - if not moved to RAM disk - eat through SSDs quite visibly. Among those are RRD and Netflow.

With my first OpnSense installation (DEC750 on 23.x) and no RAM disk, I consumed half of my enterprise-grade's lifetime in one year.

That's impressive.  I was going to recommend the OP switch to an enterprise ssd for a larger endurance.

[Taunt9930]

It is literally in the docs.

Wow, completely missed that! Thank you.

Is there a recommended setting for percentage of RAM for both?