[CLOSED] How do you run ZFS on a MiniPC?

[tessus]

This is not a ZFS usage question.

I am a bit puzzled as to how people are actually running ZFS. I am not asking how to setup ZFS or how to use it. I know all about ZFS.

I rather want to know how people find HW that supports ECC RAM.

None of the MiniPCs support ECC RAM. I have painfully learned that the hardware industry basically only supports ECC in the enterprise market. It took me weeks to find a motherboard and CPU that support ECC RAM for my virtualization server. Building a NAS with an mATX board and ECC RAM is even harder and I am still trying to find something.
But it is impossible to find a MiniPC that supports ECC RAM.

So how do people use ZFS with OPNsense?

[Patrick M. Hausen]

ZFS without ECC is still better than any other filesystem without ECC. What precisely is your problem? ZFS without ECC will not kill your data, quite the opposite.

https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/

[tessus]

I agree to disagree.

Ok, we have now one person who uses ZFS without ECC RAM. What about other people?

[dinguz]

Lots of people, including myself, run ZFS on consumer-grade hardware (without ECC). It's still a bit more reliable than UFS in my experience. My main issue with UFS was data loss or corruption in a power outage situation.

[rungekutta]

As it happens I have ECC ram but would still use ZFS even if I didn’t.

The “ZFS without ECC will destroy your data” idea was pushed very strongly on the Free/TrueNAS forums but has been pretty throughly debunked by the ZFS dev community by now. Even the TrueNAS folks seems to have accepted this. I wouldn’t worry too much. ZFS is a fine choice. Your future problems are much more likely to come from somewhere else.

[tessus]

My main issue with UFS was data loss or corruption in a power outage situation.

I use 2 UPS systems, so I don't care about power outages.

but has been pretty throughly debunked by the ZFS dev community by now.

I am not saying that another fs is safer because it is less prone to corruption. I am very well aware that e.g. ext4 can be as easily be corrupted as ZFS in case of mem errors.
The issue lies somewhere else. ZFS does not have proper recovery tools. When you have a filesystem corruption with ZFS (especially when the entire pool is affected), it's much more likely that your data is gone and non-recoverable. But I seriously don't want to discuss filesystem coding here.

However, I just had another thought. I should be fine with a corrupted ZFS on a FW appliance. Worst case scenario: I re-install and restore a config backup.

[rungekutta]

I am not saying that another fs is safer because it is less prone to corruption. I am very well aware that e.g. ext4 can be as easily be corrupted as ZFS in case of mem errors.
The issue lies somewhere else. ZFS does not have proper recovery tools. When you have a filesystem corruption with ZFS (especially when the entire pool is affected), it's much more likely that your data is gone and non-recoverable. But I seriously don't want to discuss filesystem coding here.

But what has any of that to do with ECC, which was your question (how do people run ZFS without ECC)...? Are you asking more generally if ZFS is more or less safe than other filesystems, as function of recovery tools (or not)?

Irrespective of recovery tools and the usefulness thereof, remember that ZFS has several built-in mechanisms to avoid you getting there in the first place, including checksums and redundant copies of metadata (and data if you like), scrubbing, copy-on-write at both disk management and filesystem level, etc. In any case, as for the general merits and weaknesses of ZFS (recovery tools or not), the Internet is awash with resources and needless to say ZFS has been thoroughly battle tested in very large and demanding environments for a very long time (with its roots in Solaris some 20 years ago).

However, I just had another thought. I should be fine with a corrupted ZFS on a FW appliance. Worst case scenario: I re-install and restore a config backup.

Well, yes. But again, I don't know why you worry about ZFS corruption. It should be the better choice if anything. Far more likely that some other part of your hardware breaks or that you botch up your system with a failed upgrade or whatever.

[cookiemonster]

I am not saying that another fs is safer because it is less prone to corruption. I am very well aware that e.g. ext4 can be as easily be corrupted as ZFS in case of mem errors.
The issue lies somewhere else. ZFS does not have proper recovery tools. When you have a filesystem corruption with ZFS (especially when the entire pool is affected), it's much more likely that your data is gone and non-recoverable. But I seriously don't want to discuss filesystem coding here.

But what has any of that to do with ECC, which was your question (how do people run ZFS without ECC)...? Are you asking more generally if ZFS is more or less safe than other filesystems, as function of recovery tools (or not)?

Irrespective of recovery tools and the usefulness thereof, remember that ZFS has several built-in mechanisms to avoid you getting there in the first place, including checksums and redundant copies of metadata (and data if you like), scrubbing, copy-on-write at both disk management and filesystem level, etc. In any case, as for the general merits and weaknesses of ZFS (recovery tools or not), the Internet is awash with resources and needless to say ZFS has been thoroughly battle tested in very large and demanding environments for a very long time (with its roots in Solaris some 20 years ago).

However, I just had another thought. I should be fine with a corrupted ZFS on a FW appliance. Worst case scenario: I re-install and restore a config backup.

Well, yes. But again, I don't know why you worry about ZFS corruption. It should be the better choice if anything. Far more likely that some other part of your hardware breaks or that you botch up your system with a failed upgrade or whatever.

Exactly. I've had data corruption on NASes with ZFS and non-ECC memory. It panics the kernel and only restore from backups could be done.
I also run them with ECC memory and if the same problem occurred that the memory protection didn't catch, it would be the same situation. However with ECC it has more chance to do so.
That is for Data storage systems though.

I also run my OPN with ZFS without ECC for those reasons, my small appliance doesn't support it but as you say, worst case, reinstall os and restore config. Best case, better chance of no filesystem corruption from a power loss, get boot environments, etc.
So another user with zfs and no ECC.

[tessus]

Dear moderators. Please close this topic. It's getting out of hand. I specifically mentioned that this is not a ZFS usage question. Not sure why people can't read a question properly. This means I do not need any comments on how safe it is to use ZFS w/o ECC.
No matter what someone says, using ZFS without ECC is a bad idea if you care about your data. Nobody will ever convince me otherwise. Period. (I also mentioned that ZFS is not less safe than any other fs. I said that it was much more likely that your data is gone and non-recoverable (in case of a corruption), because of the lack of proper recovery tools for ZFS.)

According to the comments people don't find proper ECC hardware either and thus use ZFS without ECC.
So for me this means back to square one.

However, I will admit that for a FW appliance, fs corruption is not the end of the world, which I have also mentioned in a previous comment.

This will be my last comment in this topic. Thanks everyone for participating.

[karlson2k]

I still don't get why are you so focused on ECC RAM?
You are talking about missing tool to fix broken filestructure on the drive.
But there are a huge number of reasons why the data can be broken.

The incomplete list includes:

• CPU broken,
• CPU microcode bug,
• Broken mainboard,
• Bad PSU,
• Bad SSD/HDD firmware,
• Bad SSD/HDD hardware,

The list could be further extended.

Any problem from the list can lead to irreversible damage of the file structures, even when using ECC RAM. The ECC RAM is not some kind of panacea that can magic solve and prevent all the problems. It is just another barrier for problems.

From a practical point of view, chances that my data will be broken by power outage with UFS are much higher that chances that data will be broken by hardware failure (with any type of filesystem).
Moreover, if I will have hardware failure, I would not try to repair the drive as even if filesystem is repaired, the data still can be broken.
In case of hardware failure that last thing I would think about is the data on the router's drive.
Most probably I would replace the broken hardware and would restore the configuration from the last known good backup.

[harshw]

I’m using a X11SCL-IF in a 1U Supermicro CSE-512 chassis. Supermicro has plenty of mini-itx and smaller motherboards with ECC RAM support. The Intel NUC9 Pro also supports ECC RAM. I’m not using ZFS at the moment but will probably move to ZFS with the next release. The other thing is to get SSDs like Micron that have Power Loss Protection- it helps for keeping your file systems safer during a power loss event. Not 100% safe but still a lot better than ordinary HDDs or SSDs

[rungekutta]

Dear moderators. Please close this topic. It's getting out of hand. I specifically mentioned that this is not a ZFS usage question. Not sure why people can't read a question properly. This means I do not need any comments on how safe it is to use ZFS w/o ECC.

With respect. Your question was

how do people use ZFS with OPNsense

… in the context of how hard it is to find ECC hardware. The problem is that this basic assertion and connection that you are making (ZFS without ECC ram = bad) is false and therefore the whole premise of your question is problematic. The responses you got were towards that end and they were factual and helpful, but then you switch topic and start talk about (lack of) recovery tools instead. Which is a completely different point that has nothing to do with either ecc ram, firewall hardware or OpnSense.

And then…

No matter what someone says, using ZFS without ECC is a bad idea if you care about your data. Nobody will ever convince me otherwise. Period. (I also mentioned that ZFS is not less safe than any other fs. I said that it was much more likely that your data is gone and non-recoverable (in case of a corruption), because of the lack of proper recovery tools for ZFS.)

… you’re back at this again. And I still don’t understand what your question is, if it isn’t about ECC ram. And, by the way, you’re wrong

Before you get too frustrated with the answers you’re getting, maybe try again and express a question more clearly and maybe you’ll get answers that you find more helpful.