Fresh install over existing UFS, import config and format as ZFS - possible?

[teosoft]

Hi all, I'm looking for advice on the following:

here's what I have:

Fully functional 21.1 setup, everything is configured over long time period. Config backups, etc.

here's what I want to do:

1) Switch to ZFS
2) Retain configuration
3) Avoid hassles

here's the plan:

Download 21.7, create a USB stick and install from it. I want both reformat the only drive in the system, from UFS to ZFS and import config and all the rest say using opnsense-importer.

I think this might not be possible because opnsense-importer reads from the UFS (/conf/*) drive first, and in my case, it's about 150MB.

Unless opnsense-importer stores it all of /conf/* in some magic location (say, ramdisk) before reformatting the entire drive to ZFS, which I don't know if it does, my plan does not seem to work.

I can experiment because I already have an identical system upgraded to 21.7, running on UFS as it was upgraded from 21.1. Thus I'm not afraid of experimenting, but the time is of value here.

TIA!

[teosoft]

Patience is not a virtue here; answering my own question.

Yes it worked for me. I followed these steps:

1. Backup your configuration - most likely you won't need it, but do it anyway.
2. Boot from USB stick with installer. Do not remove/disconnect your drive(s)
3. Upon boot, watch for import_config and import - IMPORTANT - failing this step will leave you with default configuration
4. Wait until login prompt with instructions to login as either root or installer
5. Optional - let your box run live version with your configuration. Test it. Some services won't work, I'll explain later
6. Login as installer. If you logger asa root, logout first, duh.
7. Select your keyboard layout
8. Now in in main installer window, select [ZFS] (that was my goal)
9. Do not select Import - it hangs. Waited twice for long enough to confirm hanging
10. Proceed with install. First thing you'll see is copying your configuration or something to that meaning
11. After install, remove installation media and reboot.
12. Verify that your setting were migrated - mine were
13. IMPORTANT - some plugins are not included with installer. Go to System, Status, Resolve Plugins. It will download missing plugins.
14. Voila, you're done


For me, step 13 was problematic because I run HAProxy in front of WebGUI, and because the HAProxy plugin was not installed, I had to remember how to reach WebGUI bypassing HAProxy. After I logged to the WebGUI, everything was easy.

Note to developers/maintainers - excellent migration implementation. You might want to add a menu item to a CLI, the one that shows when you ssh to the system and become root. The menu, #14 would offer to resolve plugins conflicts, just like WebGUI counterpart - that wold allow me to avoid logging on to WebGUI without functioning HAProxy.

Now I must say, I LOVE OPNSENSE! 

[Greelan]

Useful info thanks. Question: what was your rationale for wanting to move to ZFS?

[Patrick M. Hausen]

Why would you want any filesystem but ZFS in 2021?

(apart from inside VMs when the backing store is ZFS already)

[ajeffco]

Sometimes you don't need the overhead zfs brings.  Or you don't have the hardware to run it as it should be run.  There's still use cases that zfs just doesn't work in.  I love ZFS but sometimes it just isn't the right tool for the effort at hand.

And yes, you do want a backup.  I usually don't do backups on upgrades, for some reason I did this time and thank God because it saved me a lot of time and effort on rebuilding nodes when I ran into this issue with the igb driver.

Teosoft, thanks for that tho, it's good to know that it is possible to switch FS and not have to reconfigure.

[franco]

Note to developers/maintainers - excellent migration implementation. You might want to add a menu item to a CLI, the one that shows when you ssh to the system and become root. The menu, #14 would offer to resolve plugins conflicts, just like WebGUI counterpart - that wold allow me to avoid logging on to WebGUI without functioning HAProxy.

That's a very nice idea!

https://github.com/opnsense/core/issues/5136


Thanks,
Franco

[Greelan]

I finally got around to doing this UFS->ZFS transition, and I agree it is a fairly painless experience. Thanks OPNsense devs (and teosoft for sharing his tips)!

A few observations from me:

1. Know the label of the drive you are importing from. In my case it was nvd0, given I have a NVMe SSD.
2. Know your current root password. You will need that to log in as installer. Sounds obvious, but because I have root login disabled for SSH and use a separate user for SSH and webgui logins, I had forgotten the root password.
3. For a single disk installation, the only ZFS option you can use is striping (again, somewhat obvious…).
4. In addition to reinstalling plugins, you will need to re-add any non-default repos (such as mimugmail’s repo), and any OPNsense patches, and any custom configs in drop-in directories (eg for unbound nowadays, or in my case for lighttpd in /usr/local/etc/lighttpd_webgui/conf.d/).

[ad1m]

Useful info thanks. Question: what was your rationale for wanting to move to ZFS?

In my case I have often power outages and on UFS the file system would get corrupted leading to OPNsense not booting and I would have to take the OPNsense box from the closet in order to connect it to a monitor and debug. I tried multiple configurations for UFS but nothing worked. Since using ZFS the power outage and corruption situation hasn't been an issue.

[Greelan]

Fair enough. My box is on a UPS so that’s less of an issue for me, but I guess there is still a residual risk. Anyway, I’ve made the shift

[janci]

I am another happy user who succesfully transfere from UFS -> ZFS.

thanks to @teosoft for steps and @Greelan to mention about  password for installer which is same as root which is after import same as original

edit:
original verzion was 21.7 but I did flash 22.1. to usb and boot it from that, so I did also upgrade OPNSense

[sampson2269]

This worked perfectly, thanks for taking the time to do this write-up!

Patience is not a virtue here; answering my own question.

Yes it worked for me. I followed these steps:

1. Backup your configuration - most likely you won't need it, but do it anyway.
2. Boot from USB stick with installer. Do not remove/disconnect your drive(s)
3. Upon boot, watch for import_config and import - IMPORTANT - failing this step will leave you with default configuration
4. Wait until login prompt with instructions to login as either root or installer
5. Optional - let your box run live version with your configuration. Test it. Some services won't work, I'll explain later
6. Login as installer. If you logger asa root, logout first, duh.
7. Select your keyboard layout
8. Now in in main installer window, select [ZFS] (that was my goal)
9. Do not select Import - it hangs. Waited twice for long enough to confirm hanging
10. Proceed with install. First thing you'll see is copying your configuration or something to that meaning
11. After install, remove installation media and reboot.
12. Verify that your setting were migrated - mine were
13. IMPORTANT - some plugins are not included with installer. Go to System, Status, Resolve Plugins. It will download missing plugins.
14. Voila, you're done


For me, step 13 was problematic because I run HAProxy in front of WebGUI, and because the HAProxy plugin was not installed, I had to remember how to reach WebGUI bypassing HAProxy. After I logged to the WebGUI, everything was easy.

Note to developers/maintainers - excellent migration implementation. You might want to add a menu item to a CLI, the one that shows when you ssh to the system and become root. The menu, #14 would offer to resolve plugins conflicts, just like WebGUI counterpart - that wold allow me to avoid logging on to WebGUI without functioning HAProxy.

Now I must say, I LOVE OPNSENSE! 

[perholmes]

First poster and pfSense migrant here. These instructions were perfect, and OMG how easy it is to handle OPNSense. I must say, my heart rate is a lot lower. Kudos to OPNSense for making this so easy.

[franco]

thanks and welcome