Ran zpool upgrade, need to update boot code?

[os914964619]

I am running the latest Opnsense and decided to run zpool upgrade. Here was the result:

root@router:~ # zpool upgrade zroot
This system supports ZFS pool feature flags.

Enabled the following features on 'zroot':
  userobj_accounting
  encryption
  project_quota
  allocation_classes
  resilver_defer
  bookmark_v2
  redaction_bookmarks
  redacted_datasets
  bookmark_written
  log_spacemap
  livelist
  device_rebuild
  zstd_compress
  draid

Pool 'zroot' has the bootfs property set, you might need to update
the boot code. See gptzfsboot and loader.efi for details.

Is there anything I need to do, or can I reboot? I am worried that I'm going to brick my opnsense install by not updating "the boot code".

[Patrick M. Hausen]

You should update the boot code. The details are in the mentioned man pages. 

OK, let's start with the output of zpool status and gpart show, please. Do you know if your system boots legacy or EFI?

[rackg]

What feature or flag you need desperately ? Playing with zpool upgrade without backup is something not recommended.
I would definelty not upgrade zroot atleast. 

I am running the latest Opnsense and decided to run zpool upgrade. Here was the result:

root@router:~ # zpool upgrade zroot
This system supports ZFS pool feature flags.

Enabled the following features on 'zroot':
  userobj_accounting
  encryption
  project_quota
  allocation_classes
  resilver_defer
  bookmark_v2
  redaction_bookmarks
  redacted_datasets
  bookmark_written
  log_spacemap
  livelist
  device_rebuild
  zstd_compress
  draid

Pool 'zroot' has the bootfs property set, you might need to update
the boot code. See gptzfsboot and loader.efi for details.

Is there anything I need to do, or can I reboot? I am worried that I'm going to brick my opnsense install by not updating "the boot code".

[Patrick M. Hausen]

Any fresh install will create a zpool with all these features enabled, so why not upgrade your existing one if only to get rid of that message in `zpool status`?

I prefer consistent state.

[meyergru]

I know I revive a very old thread, however, a word of warning and advice is due:

Do not upgrade your boot zpool unless you have prepared the system first - I found out the hard way today!

I now know for a fact that OpnSense does not update the boot settings for either UEFI or for BIOS booting after initial installation. You can verify yourself by looking at /boot/loader.efi (which is the updated current UEFI boot loader) and comparing it to /boot/efi/efi/boot/bootx64.efi and /boot/efi/efi/freebsd/loader.efi, the two latter of which are created only once at initial installation, thus /boot/loader.efi will most likely differ (do not worry if the newer files are actually smaller).

Same goes for the files in the gptzfsboot partition for BIOS booting.

For me, it was with the new FreeBSD 14.1 kernel on 24.7.x that had a few new zpool features:

Code: [Select]

# zpool upgrade zroot
This system supports ZFS pool feature flags.

Enabled the following features on 'zroot':
  edonr
  zilsaxattr
  head_errlog
  blake3
  block_cloning
  vdev_zaps_v2

Pool 'zroot' has the bootfs property set, you might need to update
the boot code. See gptzfsboot(8) and loader.efi(8) for details.

The incompatible feature that stopped booting for me was head_errlog - just because the zpool was not recognized by the old boot file.

To avoid this happening, say your boot drive is on da0, you will have to do the following for UEFI (before you try to reboot!):

Code: [Select]

cp /boot/loader.efi /boot/efi/efi/boot/bootx64.efi
cp /boot/loader.efi /boot/efi/efi/freebsd/loader.efi

and for BIOS (you can do both just in case):

Code: [Select]

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 da0

Replace "da0" with your specific boot disk (find out the name with goem disk list) and note "-i 2" instead of "-i 1" from the documentation, because the BIOS boot partition is 2 instead of 1 for OpnSense, because it is set up for dual booting (partition 1 is for UEFI boot).


If the worst has happened and your system does not boot any more, you can either reinstall OpnSense from scratch or create a FAT32/VFAT USB stick with a correct loader.efi and copy that to the EFI partition into the correct locations (this is tedious, however). IDK how to fix it with BIOS boot.

[Patrick M. Hausen]

If the worst has happened and your system does not boot any more, you can either reinstall OpnSense from scratch or create a FAT32/VFAT USB stick with a correct loader.efi and copy that to the EFI partition into the correct locations (this is tedious, however). IDK how to fix it with BIOS boot.

Boot a FreeBSD 14.1 CD/DVD image, escape to shell, use the same gpart command you gave above to install the boot loader.

Similar for EFI:

Code: [Select]

mount -t msdosfs /dev/da0p1 /mnt
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd/loader.efi
umount /mnt

[meyergru]

If the worst has happened and your system does not boot any more, you can either reinstall OpnSense from scratch or create a FAT32/VFAT USB stick with a correct loader.efi and copy that to the EFI partition into the correct locations (this is tedious, however). IDK how to fix it with BIOS boot.

Boot a FreeBSD 14.1 CD/DVD image, escape to shell, use the same gpart command you gave above to install the boot loader.

Similar for EFI:

Code: [Select]

mount -t msdosfs /dev/da0p1 /mnt
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd/loader.efi
umount /mnt

Yup. After having someone at the remote site connect a display, keyboard and having no FreeBSD iso (and no CD-ROM either). I had to prepare a USB stick for download. Then, they had no internet to download that. Then, they had a Mac and no USB stick... been there, done that - not funny in any case.

I opened a feature request, although I know it was my own fault...

[Patrick M. Hausen]

I opened a feature request, although I know it was my own fault...

Are you sure we want that? OPNsense does not necessarily run with the same partition scheme everywhere. Thanks to the bootstrap install method there could be any fancy partition and zpool layout.
Think e.g. Hetzner: Linux rescue system --> depenguinator --> opnsense-bootstrap.sh

Honestly I for one would rather install without the legacy boot partition and EFI only if the system can EFI boot. If you find both, you never know which one to update so you always have to do both just in case. The admin should set the one correct way at installation time, IMHO.

That very topic is currently discussed on the freebsd-arm mailing list. There seems to be no "one size fits all" solution, unfortunately.

Kind regards,
Patrick

P.S. Of course we could define and document that OPNsense will always have an EFI partition in p1 and a legacy boot partition in p2 and then just automatically update. And if you insist on rolling your own installation you better stick to that partition scheme. I would be perfectly fine with that. Truenas (for example) is no different: always refer to drives by partition UUID when manipulating your pool(s) - if you don't, bad things happen.

P.P.S. That still does not solve the case of mirrored or otherwise redundant installations.

[franco]

IIRC FreeBSD wanted to work towards being able to handle the boot code updates at some point. For EFI they were already adding /boot/efi to the running systems by default in some prior upgrade but there appears to be no boot partition upgrade code yet and it's harder for even older system that will not have all the mount points set. It's a slow grind and the risks are quite real either way.

I don't necessarily want this solved on our end before knowing what the actual plan is in FreeBSD (something within pkg-base perhaps?). We will have to wait and see...


Cheers,
Franco

[meyergru]

Yes, probably you need to have some intelligence to do this to fit all needs...

[cookiemonster]

I for one like that the images provided by OPN have both EFI and legacy BIOS so a single download/image can be used on either type of system. That said, I agree a documentation could be enhanced to better inform, because it is very true that once installed, it's hard to know in an emergency and needs to update both.
I suppose the alternative to provide each image X 2, one for BIOS and one for EFI could deal with it in another way.

[meyergru]

You only need to update either BIOS or UEFI, because your specific system only uses one. But just for good measure, I would always update both types.

[franco]

I for one like that the images provided by OPN have both EFI and legacy BIOS so a single download/image can be used on either type of system. That said, I agree a documentation could be enhanced to better inform, because it is very true that once installed, it's hard to know in an emergency and needs to update both.
I suppose the alternative to provide each image X 2, one for BIOS and one for EFI could deal with it in another way.

Not sure what you meant to say but we actually do have both images and installation being dual-boot? Serial, vga, dvd plus the hybrid ZFS or UFS install modes...


Cheers,
Franco

[doktornotor]

But just for good measure, I would always update both types.

For good measure, I'd never run zpool upgrade on OPNsense because it brings virtually nothing useful, only potentially makes your system not bootable any more.

[meyergru]

After you install fresh, you will have exactly those zpool features active anyway, useful or not.