OPNsense Forum

English Forums => Hardware and Performance => Topic started by: ozpac on April 18, 2024, 07:13:44 PM

Title: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: ozpac on April 18, 2024, 07:13:44 PM
My ZFS root accumulated some file damage and then refused to mount.
I will rebuild the filesystem from scratch but want to increase resiliency by making use of ZFS's self healing abilities which requires mirrored data.
I'm using a Protectli device which only has one drive bay (a mini pcie port for mSATA cards).
My idea is to split that 128gb drive into two 64GB partitions then make the OPNSense system root a mirrored zpool across the two.

I will also run daily snapshots to external drives and so on, but I want better prevention rather than just better cures.

Is this a good plan?
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: Patrick M. Hausen on April 18, 2024, 07:21:37 PM
How do you replace the broken partition while keeping the other one? Because that is what "self healing" means. You replace the broken disk. ZFS "resilvers" the data until redundancy is restored.

Does not make much sense to me.
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: ozpac on April 18, 2024, 07:33:46 PM
As I understand it, ZFS computes and stores checksums for everything (blocks?). It uses these to determine whether the block is in tact or damaged. If damaged then it will refer to the mirror to see if that block is in tact. It will then automatically write the undamaged block over the damaged one - thus self healing.  With only one copy of the data then self healing isn't possible.
Isn't that accurate?

In other words, this is the only way I can defend against bitrot, on this hardware. If I can do more I would love to do so.
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: Patrick M. Hausen on April 18, 2024, 07:36:12 PM
A broken block stays broken until you perform a resilver operation. Usually after you swapped the broken device.
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: ozpac on April 18, 2024, 07:40:36 PM
Ah I see, that makes sense.
How about bitflips (eg cosmic ray)? This would mitigate them wouldn't it?
I gather they are very rare but nevertheless, would my scheme work?
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: Patrick M. Hausen on April 18, 2024, 07:44:09 PM
Most certainly yes. You could also create the zpool with copies=2 or copies=3

https://jrs-s.net/2016/05/02/zfs-copies-equals-n/

Mark the second paragraph with the laptop example.
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: ozpac on April 18, 2024, 07:53:43 PM
Incredibly helpful info, thank kindly! I appreciate your efforts.
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: hushcoden on April 21, 2024, 05:28:53 PM
Question from a newbie: is the ZFS's self healing enabled by defauld having installed OPNsense with two drives (mirror) ?

Tia.
Title: Re: Enable ZFS self healing by mirroring across 2 partitions of same SSD?
Post by: Patrick M. Hausen on April 21, 2024, 05:33:23 PM
There is no healing without operator intervention. You can enable regular scrubs via cron jobs, though.