Installation on ZFS

[danb35]

I'm looking at moving from pfSense to OPNsense, but I'm a definite fan of pfSense's root-on-ZFS installation option.  I don't see that in OPNsense, unless it's hidden somewhere--it's certainly not as prominent as it is in the FreeBSD 11.2 installer.

But OPNsense has the bootstrap installer, so I can install FreeBSD 11.2 on ZFS and then run that script, right?  I've tested it, and it seems to work--at least to the point of being able to pull up the OPNsense web GUI.  But I'm a little concerned, since the documentation says:

What it will also do is turn a supported stock FreeBSD release into an OPNsense installation, given that UFS was used to install the root file system.

Am I setting myself up for trouble here?  Is this an obsolete caveat in the manual?  Or is something else going on?

[pouakai]

It's worth reading this forum post https://forum.opnsense.org/index.php?topic=12267.0, particularly posts 14 and 15.

I installed to ZFS  a few weeks ago, and the same note on the documentation gave me some concerns, but all went smoothly.

You do have to use the same FreeBSD version as Opnsense (that's 11.2 for for 20.1).  Newer versions of FreeBSD work until you reboot, giving a mount error.  I think this is because the ZFS package in Opnsense is older than the installed filesystem in later versions of FreeBSD and therefore incompatible.

Poking around the directory structure, Opnsense seems to have all the support for installed ZFS, and it's working nicely.  Thanks to the developers!

[danb35]

Thanks, I'd missed that thread in my searching--though I see there's the standard anti-ZFS FUD there too...  Edit: I should clarify that.  It seems a lot of people, including one poster in that thread, believe that ZFS' hardware requirements are the same as FreeNAS', which just isn't true.  No, ZFS doesn't need 8GB of RAM (though yes, it will use as much as you have for caching, though it will release it if it's needed by other processes).  No, it doesn't need ECC RAM.

I installed it Tuesday afternoon as described--install FreeBSD 11.2 on ZFS then run the script.  There were a few obstacles (mostly dealing with the serial console on the SG-2440 box and trying to import configuration from pfSense), but it's now up and running and seems to be working well.  But I'd still like to know why that note's in the docs...

[franco]

Sorry, this was copied from update.git README but that has been out of date for a few years as well it seems.

https://github.com/opnsense/update/commit/3298b03636
https://github.com/opnsense/docs/commit/104de351009


Cheers,
Franco

[Patrick M. Hausen]

OMG ... must install again I fear 

@danb35, nice to meet you, Ambassador.

[danb35]

nice to meet you, Ambassador.

I have, of course, always been here.  Though I guess that doesn't make as much sense without the avatar.

[Moofo]

For folks considering doing it the following worked perfectly for me on WatchGuard XTM 5 upgraded with a quad core cpu, 4Gb RAM and an SSD. I did not patch the BIOS.

 

• Install FreeBSD 12.1 on ZFS
• Download and start the OpnSense Bootstrap

I had absolutely no issues.

I did this for the sole purpose of not messing the file system at every power outage.

[ejprice]

I'm a huge fan of ZFS. It is an awesome, reliable, performant filesystem with undeniable resiliency. It is way better than UFS and pretty much every single FS out there. I used ZFS everywhere I can.

Now that I see a clear path to use ZFS on OpnSense, I'm already thinking of ways to do it - because ZFS is that awesome, and most importantly, resilient. 

However, to use ZFS without ECC memory is a bad idea. I won't get into the why(s) as you can just search for the plethora of articles on it. Yes, you can do it - but no, you really, really don't want to.

Also, to get the most bang for your buck with ZFS you should have a separate, fast, small SLOG device for the ZIL. To keep things cost-effective you can combine the SLOG device and the L2ARC on the same NVME and still expect reasonable performance.

Hope that helps.

[Patrick M. Hausen]

There's a plethora of articles on why you should never ever run ZFS whitout ECC memory, because it will destroy your data on disk. Unfortunately all of these articles are a load of bull...

The "scrub of death" is a myth.

ZFS whihtout ECC is still better than any other filesystem whithout ECC with respect to preserving your data.

Period.

Read about it here:
https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/

Attend the ZFS developer meetings - they are public.

Listen to bsdnow.tv


But please stop spreading unsubstantiated FUD ...

[ejprice]

There's a plethora of articles on why you should never ever run ZFS whitout ECC memory, because it will destroy your data on disk. Unfortunately all of these articles are a load of bull...

The "scrub of death" is a myth.

ZFS whihtout ECC is still better than any other filesystem whithout ECC with respect to preserving your data.

Period.

Read about it here:
https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/

Attend the ZFS developer meetings - they are public.

Listen to bsdnow.tv


But please stop spreading unsubstantiated FUD ...

Actually, I'm subscribed to the OpenZFS mailing list and read the posts every day. The most frequent troubleshooting question asked when someone has a bizarre problem is "Do you have ECC memory?"

So, in your opinion, it's FUD. I disagree. I think the additional resiliency ECC provides, particularly to ZFS, is worth the small extra outlay. I suggested readers READ on their own and make their own decision - which is certainly not FUD but encouraging people to do their own research and form their own opinion.

[idimitro]

Hi, I am planning to move to opnSense and I wan to run it on ZFS.
I wondering if I am losing the "Hardened BSD" aspects of the opnSense by deploying it with the bootstrap script, or will the system be hardened post installation?
Thanks.