OPNsense Forum
English Forums => General Discussion => Topic started by: Aadolf on January 14, 2017, 04:10:45 pm
-
hi
I think there was a feature in 17.1 road-map and that was "ZFS rc mount awareness" but now there isn't such a feature.
What was that (ZFS rc mount awareness ) ?
thanks
-
Hi,
Yes, you are right. I tracked down several FreeBSD issues in preparation for 17.1 that took most of my time planned for other things amongst "zfs rc mount awareness".
It simply mean if you had a working ZFS custom install, OPNsense would boot fine on it, which it currently doesn't.
Since the installer doesn't support ZFS installation yet, the former was really only for advanced users and made not a lot of sense in another way than the prerequisite for proper ZFS support.
With all the work having happened for FreeBSD 11.0 in the past months, full ZFS support shifts to the 17.7 roadmap.
I will note this in the 17.1-RC release notes.
Cheers,
Franco
-
Good news.
Thanks.
-
So any progress on ZFS yet?
-
It‘s true, being postponed up until now.
Bootstrapping into a FreeBSD with ZFS would be a good first start, but it’s impractical for most users as the installer should also be capable of installing OPNssense on ZFS, which is the larger and more tedious work.
That being said, OPNsense can’t boot ZFS yet, so please don’t try to bootstrap from a FreeBSD on ZFS.
I don’t know how much time there still is, 18.1 is quite full in terms of goals and code reworks but I would like to ask you to keep prodding, because that has shown to be quite effective.
Sorry for not being able to be more assertive on the matter.
Thank you,
Franco
-
Can you elaborate why one that is, other than not supported yet. I ask as im new to opnsense but very experienced with freebsd, and zfs builds. Initially looking around at what is available in the base userland, I suspect I could get it going in and incorporate boot environments, unless there is something hidden in there. All the libraries, kernel modules and loaders appear to be there, which is a good start. After all it does look like opnsense is a fairly bog standard freebsd build with tweaks, config management and a gui.
-
It was state here very recently: https://forum.opnsense.org/index.php?topic=8521.0
Translated:
Requires time for development. Every now and then somebody brings up a new networking feature they want to merge and that is subsequently prioritised... :)
We need to do two things:
1. ZFS Boot Awareness: With a FreeBSD on ZFS you can install OPNsense using opnsense-bootstrap, but the system won't boot because /usr/local/etc/rc is not aware of ZFS and the kernel module is missing.
2. When 1. is done the installer needs to gain a ZFS installation mode.
Cheers,
Franco
-
Whatever makes ZFS happen. Been waiting a long time.
-
Ok, here's some progress then: https://github.com/opnsense/core/issues/2386
Cheers,
Franco
-
Nice! Thank you Franco! Been looking forward to this as well. Great feature to have for areas that aren't stable on power and need to resume with a minimum of data loss, UFS is not too friendly in that regard.
-
Yikes, yes, UFS recovery is
-
Apart from that latent joke earlier, feel free to try this bootstrap method into the latest development version with a standard amd64 FreeBSD 11.1 ZFS install (do not deviate from the defaults: auto, stripe):
-- REDACTED, see later replies --
Cheers,
Franco
-
PS: Careful on updates, 18.1.7 will not retain this support, but I'm positive it'll be in 18.1.8.
-
# pkg install ca_root_nss
# fetch https://raw.githubusercontent.com/opnsense/update/master/bootstrap/opnsense-bootstrap.sh
# sh ./opnsense-bootstrap.sh -t opnsense-devel
So, if I wanted to test an OPNsense ZFS build, I would install a FreeBSD 11.1 ZFS build, then run these commands?
-
Yes, but make sure it's amd64 and all defaults for ZFS (striped). More modes may work, but they haven't been tested. Encryption doesn't work for sure.
These changes will all be in 18.1.8 from the looks of it to make the required installer changes easier.
Cheers,
Franco
-
This is awesome!
-
Thanks for the information. I am confused though, why was the standard BSD bootstrap deviated from. Was this an opnsense design decision or was it inherited from the pfsense fork? If the latter does anyone know the reason?
It was state here very recently: https://forum.opnsense.org/index.php?topic=8521.0
Translated:
Requires time for development. Every now and then somebody brings up a new networking feature they want to merge and that is subsequently prioritised... :)
We need to do two things:
1. ZFS Boot Awareness: With a FreeBSD on ZFS you can install OPNsense using opnsense-bootstrap, but the system won't boot because /usr/local/etc/rc is not aware of ZFS and the kernel module is missing.
2. When 1. is done the installer needs to gain a ZFS installation mode.
Cheers,
Franco
-
Hi there,
"standard BSD bootstrap" is highly confusing. ZFS is only available in FreeBSD, and assuming by bootstrap you mean bootup of FreeBSD.
M0n0wall was forked from FreeBSD pre-ZFS and since used a custom bootup sequence partially based on PHP. This was inherited by pfSense and subsequently OPNsense.
As far as I'm aware M0n0wall never had ZFS support. pfSense gained ZFS boot support, but until we forked it was never in state where the shipped installer was able to install a ZFS. When the time came to fork pfSense we removed the ZFS boot support because we couldn't verify it either way because we also inherited the lack of ZFS installer support.
Later pfSense switched from the original bsdinstaller to the bsdinstall in FreeBSD which brought them ZFS installer support. That work is still pending with us, although I think we will stick with the bsdinstaller for the foreseeable future and modify it accordingly.
Cheers,
Franco
-
Just installed it. Works great! Thanks
-
:)
-
18.1.8 is out with the ZFS changes included, so now I'd advise use of...
# pkg install ca_root_nss
# fetch https://raw.githubusercontent.com/opnsense/update/master/bootstrap/opnsense-bootstrap.sh
# sh ./opnsense-bootstrap.sh
as documented via https://github.com/opnsense/update#opnsense-bootstrap
Cheers,
Franco
-
Works well with 2.1.9.
I can attest to it working on a raidz2 pool with six drives. I stacked three dual m.2 to sata adapters (also can be dual mSATA but not both at the same time) with Sata M.2 drives. The motherboard is a six SATA port eight Intel Nic Jetway with an i7.
-
18.1.8 is out with the ZFS changes included, so now I'd advise use of...
# pkg install ca_root_nss
# fetch https://raw.githubusercontent.com/opnsense/update/master/bootstrap/opnsense-bootstrap.sh
# sh ./opnsense-bootstrap.sh
as documented via https://github.com/opnsense/update#opnsense-bootstrap
Cheers,
Franco
I should mention that when installing BSD 11.1, the networking is not as user friendly as the OPNsense installer, you have to choose Unbound before rebooting after BSD installs or you won't be able to install the package manager and OPNsense. I don't recall seeing the other resolver as an option. Once OPNsense is installed, do not forget that the root password is opnsense at this point, not what you chose during the BSD install. This would be the time to change your root password and networking, then you are up and running.
-
Hello, I just wanted to add that this is epic, OPNsense + ZFS + Boot Environments(beadm). 8)
I'm really happy this solution is working as expected on top of vanilla FreeBSD 11.1, brings me peace of mind that I'm backed by Boot Environments before upgrades/system wide changes.
Regards
-
Hi,
beadm is already in our packages repository for manual install. If somebody wants to write a plugin that would be fantastic.
Cheers,
Franco
-
Hello, this may seems a bit redundant for many as "beadm" commands are extremely simple, but I've created a cdialog driven little tool to help mostly newbies to work with Boot environments from a TUI manager for convenience. ::)
I posted brief install instructions HERE (https://nas4free.org/forums/viewtopic.php?f=71&t=13761#p85237) though, I'm also currently using it on OPNsense.
Regards
-
Thank you, that's great. Maybe we can use it as a guide to plugin development some time during 18.7.x ? :)
Cheers,
Franco
-
That would be great, I will be up for any testing when it's time though, for now I'm more than happy that my hardware is working exceptionally well with OPNsense/ZFS. 8)
Regards
-
Sorry to poke here again, I just wanted to post my latest simple beadm TUI wrapper, it now supports full Boot Environments replication to .xz compressed files, this creates a replica(zfs send) of your last saved Boot Environment/Snapshot(I use zfsnap for rolling snapshots), then piped to xz regardless of the ZFS RAIDX used.
This can be later restored if you lost your boot drive(s) or simply want to replicate pre-configured setups to alternate hardware/VM's for convenience, note that this tool can backups your BE's to same boot drive as well(I use a 2.5" 250GB HDD w/o issue) and later exported to whatever location or even your desktop with simple tools like FileZilla/WinSCP through SSH, but is not limited to, you can edit the tool and save BE's to a network location as well.
This might help someone with PHP skills to make a similar(optional) plugin I think, Bemanager at GitHub (https://github.com/JRGTH/bemanager).
Images and sample config file HERE (http://jmrpub.serveo.net/Share/Devel/FreeBSD/ZFS/BE_TUI_Manager/)
Regards
-
For multiple personal reasons, I'm not able to finish the ZFS installer support for the initial 18.7.
Cheers,
Franco
-
At least it can boot it and thankfully OPNsense installs easily over BSD thanks to the script.
-
Indeed, have just done so myself. Very straight forward.
-
bemanager discussion can be found here now:
https://www.xigmanas.com/forums/viewtopic.php?f=66&t=13817&p=85527&hilit=beadm#p85527
-
Just a heads up. When using ZFS, password recovery from the live image does not work any more. The script can't mount the partitions.
-
The installer does not have ZFS support yet, yes.
-
What's the state in Januar 2019 ?
-
Full ZFS support. No ZFS installer. Since 19.1 you can import your settings from a previous ZFS pool, e.g. for testing updates in live mode before upgrading your ZFS install.
Cheers,
Franco
-
Any plans for ZFS aware installer for 20.1? :D
-
Plan is 19.7. Community contributions is zero. ;)
Cheers,
Franco
-
Is ZFS installer resp. bsdinstall implemeted in 19.7 now?
-
Plan is 19.7.
Cheers,
Franco
hi Franco, this doesn't seem to be in 19.7, do you have a different plan for it?
-
It's on the eternal wish list. Right now there are delays with the move to 12.1 so it's likely postponed to 20.7 or beyond.
I'm partly to blame because I don't work for OPNsense so we can't make this happen on a more definitive basis.
Cheers,
Franco
-
thanks!