Dear all,
may you can support, I'm a bit confused.
I've installed the most recent version of opnsense with ZFS. Main reason is the possibility of snapshots.
There is one pool created "zroot", I try to create and rollback a snapshot but without success, may you have some ideas. Here some code (clean install for test purposes)
root@OPNsense:/ # zfs list -t all
NAME USED AVAIL REFER MOUNTPOINT
zroot 904M 48.5G 88K /zroot
zroot/ROOT 900M 48.5G 88K none
zroot/ROOT/default 900M 48.5G 900M /
zroot/tmp 152K 48.5G 152K /tmp
zroot/usr 352K 48.5G 88K /usr
zroot/usr/home 88K 48.5G 88K /usr/home
zroot/usr/ports 88K 48.5G 88K /usr/ports
zroot/usr/src 88K 48.5G 88K /usr/src
zroot/var 1.36M 48.5G 88K /var
zroot/var/audit 88K 48.5G 88K /var/audit
zroot/var/crash 88K 48.5G 88K /var/crash
zroot/var/log 956K 48.5G 956K /var/log
zroot/var/mail 88K 48.5G 88K /var/mail
zroot/var/tmp 88K 48.5G 88K /var/tmp
root@OPNsense:/ # echo "test1" > test.txt
root@OPNsense:/ # cat test.txt
test1
root@OPNsense:/ # zfs snap -r zroot@snap1
root@OPNsense:/ # echo "test2" > test.txt
root@OPNsense:/ # cat test.txt
test2
root@OPNsense:/ # zfs list -t all
NAME USED AVAIL REFER MOUNTPOINT
zroot 904M 48.5G 88K /zroot
zroot@snap1 0 - 88K -
zroot/ROOT 900M 48.5G 88K none
zroot/ROOT@snap1 0 - 88K -
zroot/ROOT/default 900M 48.5G 900M /
zroot/ROOT/default@snap1 56K - 900M -
zroot/tmp 152K 48.5G 152K /tmp
zroot/tmp@snap1 0 - 152K -
zroot/usr 352K 48.5G 88K /usr
zroot/usr@snap1 0 - 88K -
zroot/usr/home 88K 48.5G 88K /usr/home
zroot/usr/home@snap1 0 - 88K -
zroot/usr/ports 88K 48.5G 88K /usr/ports
zroot/usr/ports@snap1 0 - 88K -
zroot/usr/src 88K 48.5G 88K /usr/src
zroot/usr/src@snap1 0 - 88K -
zroot/var 1.43M 48.5G 88K /var
zroot/var@snap1 0 - 88K -
zroot/var/audit 88K 48.5G 88K /var/audit
zroot/var/audit@snap1 0 - 88K -
zroot/var/crash 88K 48.5G 88K /var/crash
zroot/var/crash@snap1 0 - 88K -
zroot/var/log 1.00M 48.5G 956K /var/log
zroot/var/log@snap1 72K - 956K -
zroot/var/mail 88K 48.5G 88K /var/mail
zroot/var/mail@snap1 0 - 88K -
zroot/var/tmp 88K 48.5G 88K /var/tmp
zroot/var/tmp@snap1 0 - 88K -
root@OPNsense:/ # zfs rollback -r zroot@snap1
root@OPNsense:/ # cat test.txt
test2
After the rollback my assumption was that "test1" is the output?
My goal is to create before any update an snapshot to be able to rollback to the previous state if anything unexpected happens.
br
There is no recursive flag for zfs rollback. You need to rollback all the individual filesystem manually.
Specifically in your case - probably - zroot/ROOT/default@snap1.
The "-r" flag for zfs rollback does not mean "recursive". Read `man zfs-rollback` for details.
I frequently use something like this:
zfs list -t snap -h | awk '/@snap1/ { printf "zfs rollback %s\n", $1 }' | sh
But ...
you might want to look into boot environments. They take care of the snapshots and allow you to give fancy names to your various versions, even boot into past ones if you have console access.
# list current BEs
bectl list
# assume we are running 21.7 and the major update to 22.1 is waiting
# rename "default" to "21.7"
bectl rename default 21.7
# create new BE for the new version
bectl create 22.1
# activate new BE for next reboot, then reboot into it
bectl activate 22.1
reboot
# now perform UI update
# after reboot 2 BEs will be present: 21.7 and 22.1 - you can pick them at the boot loader prompt if necessary
bectl list
You can do the same with minor versions, of course. All the work is already been done. Enjoy ;)
Thanks, great! :)
Quote from: pmhausen on October 04, 2021, 09:58:32 PM
But ...
you might want to look into boot environments. They take care of the snapshots and allow you to give fancy names to your various versions, even boot into past ones if you have console access.
# list current BEs
bectl list
# assume we are running 21.7 and the major update to 22.1 is waiting
# rename "default" to "21.7"
bectl rename default 21.7
# create new BE for the new version
bectl create 22.1
# activate new BE for next reboot, then reboot into it
bectl activate 22.1
reboot
# now perform UI update
# after reboot 2 BEs will be present: 21.7 and 22.1 - you can pick them at the boot loader prompt if necessary
bectl list
You can do the same with minor versions, of course. All the work is already been done. Enjoy ;)
Holy crapperjacks that's awesome! Was bectl made by/for OPNSense or is this a FreeBSD tool that I have yet to come across? Well done whoever made it, like a cross between installing a new Linux kernel and making a VM snapshot, hah!
Is it worth to build a plugin around it?
@pmhausen: is there a way to define a fallback if Default doesnt boot?
Quote from: tmanok on October 07, 2021, 06:09:14 AM
Holy crapperjacks that's awesome! Was bectl made by/for OPNSense or is this a FreeBSD tool that I have yet to come across? Well done whoever made it, like a cross between installing a new Linux kernel and making a VM snapshot, hah!
Standard FreeBSD since ... well almost since ZFS. There was the beadm uitility written in shell, which mimicked the Solaris tool of the same name. Was replaced by bectl written in C and using libzfs in 12.x or 11.x, even, can't remember.
Kind regards,
Patrick
Quote from: mimugmail on October 07, 2021, 07:58:00 AM
Is it worth to build a plugin around it?
@pmhausen: is there a way to define a fallback if Default doesnt boot?
You can always set a BE to "boot once" and if that fails have a sneaker admin power cycle the machine and it will boot into the last active one, which is hopefully "known good".
So probably:
bectl create new-version
bectl activate -t new-version # only activates for one boot
shutdown -r now
# do update
bectl activate -t new-version
shutdown -r now
# check if it works, if ok, then
bectl activate new-version # activate permanentlyYou might want to look at TrueNAS CORE and the System > Boot section of the UI for inspiration ;)
Hearing about boot environments is a revelation to me too! It's prompted me to do some more reading, and it seems amazing. Another reason why I'm glad I recently did the conversion to ZFS for my OPNsense installation
I wonder if the -r flag is needed to be able to go back to a previous system version.
eg bectl create -r 22.1
Anyone knows?
Patrick described the workfkow in this thread:
https://forum.opnsense.org/index.php?topic=25540.msg122750
Quote from: senser on January 30, 2022, 01:33:03 PM
I wonder if the -r flag is needed to be able to go back to a previous system version.
eg bectl create -r 22.1
In general - no. The dataset layout of FreeBSD on ZFS is carefully tailored so that everything impacted by an update is in one single dataset - the boot environment.
Things like users' home directories and all of /var/* are excluded, because they might contain valuable continuously changing application data that you want to keep even in case you do a rollback of an update gone wrong.
In the OPNsense case there might occacionally be a problem with a particular service, when data structures in /var/* are changed by an update and the older version cannot cope with that. In that case you need to investigate individual services.
But nothing keeps you from doing an extra `zfs snapshot ...` for all the datasets you might want to roll back in addition to the boot environment.
HTH,
Patrick
Quote from: pmhausen on January 30, 2022, 03:59:45 PM
In general - no. The dataset layout of FreeBSD on ZFS is carefully tailored so that everything impacted by an update is in one single dataset - the boot environment.
Things like users' home directories and all of /var/* are excluded, because they might contain valuable continuously changing application data that you want to keep even in case you do a rollback of an update gone wrong.
Thats pretty cool! Thanks for the info! :)