How to rebuild RAID 1 setup (ZFS)

[peto59]

Hello.
I have a RAID 1 setup running on ZFS, and one of the disks died this morning.
How do I rebuild the RAID?
I do know how to rebuild ZFS raid. What I'm more interested in is what to do with other partitions.
I seem to have EFI, boot and swap partitions.
gmirror does not show any output.
Output of gpart show:

Code: [Select]

gpart show
=>       40  351651808  ada0  GPT  (168G)
         40     532480     1  efi  (260M)
     532520       1024     2  freebsd-boot  (512K)
     533544        984        - free -  (492K)
     534528   16777216     3  freebsd-swap  (8.0G)
   17311744  334340096     4  freebsd-zfs  (159G)
  351651840          8        - free -  (4.0K)
I'm thinking about restoring partitions with

Code: [Select]

gpart backup /dev/ada0 | gpart restore -F /dev/ada1and then dd-ing the contents.
Will it work? I believe it will copy over UUIDs as well, which might pose a problem.
I would appreciate it if the resulting setup also had data consistency over updates.
What are the correct steps here?

Thanks!

[Patrick M. Hausen]

1. gpart backup <old disk> | gpart restore <new disk>
2. zpool replace ...
3. dd freebsd-boot and efi
4. AFAIR the installer does not create mirrored swap, you would have to setup that yourself

[peto59]

Thank you for your fast reply.
So, I was correct in assuming that dd is enough.
The swap is probably irrelevant in my case, but I might look into it.
If you have the time, would you be so kind as to explain how this is enough for the OPNsense to update EFI and boot partitions on both disks in case of a bootloader update?
And does not dd also copy UUIDs, which results in a mess? Does dd even copy over UUIDs? Admittedly, I am used to Linux, so I might not pick up on the slight differences between Linux and Unix.

[Patrick M. Hausen]

It is not in case of boot loader updates. I assumed you have valid boot loaders on ada0. These can be copied to another disk with dd.

In case you don't:

1. For the freebsd-boot partition

Code: [Select]

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
2. For EFI

Code: [Select]

newfs_msdos /dev/ada0p1
mount -t msdosfs /dev/ada0p1 /mnt
mkdir -p /mnt/efi/boot /mnt/efi/freebsd
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd/loader.efi
umount /mnt
When you are not sure which method your hardware uses to boot (and even in case you are because you might move the disk to a new system) I advise to update/create both.

[peto59]

This is most interesting. I had no idea it was this easy in OPNsense. Thank you.
But you were right in your assumption. The box is currently running in full operation. I am just waiting for the spare disk to arrive (an oversight on my part).

I have not explained myself clearly. You have fully answered my original question. What I was curious about in my second question is how the updates to boot/EFI partitions are handled if they are not in a RAID.

In my experience in Linux, if the contents of the EFI partition changed, I had to copy over the changes manually or set up a RAID over them.

The dd command is not optimal for this (in Linux) because it copies over the UUIDs, which creates a mess down the line.

Is this different in OPNsense/FreeBSD? Do the contents of the EFI partition not change? Or is there some script which automates the process of syncing changes during an update? Or does OPNsense not use UUIDs to identify disks?

[Patrick M. Hausen]

In my experience in Linux, if the contents of the EFI partition changed, I had to copy over the changes manually or set up a RAID over them.

Same for FreeBSD

The dd command is not optimal for this (in Linux) because it copies over the UUIDs, which creates a mess down the line.

If you use dd to copy a partition's content that should not mess with the UUID in the partition table proper.

Do the contents of the EFI partition not change?

You might need to update the boot loader (regardless if legacy or EFI) when you perform a release upgrade and in the process upgrade your zpool. The boot loader needs to support all zpool features which are active.

The good thing about FreeBSD is that the boot loader auf release X.Y can always boot a pool of release X.Y - contrary to Grub I am told.

Or is there some script which automates the process of syncing changes during an update?

Nope, manual operator duty. There is no mechanism upgrading the boot loader after initial installation ever. Yet - see above - zpool upgrades might require that.

Or does OPNsense not use UUIDs to identify disks?

Traditionally FreeBSD (and OPNsense) just use device and partition numbers like ada0p1. You can use UUIDs and e.g. TrueNAS CORE does by default. OPNsense to my knowledge sticks to devices.

[peto59]

This was most enlightening. I entered this with some assumptions from Linux, which are incorrect here. I have read more about the boot and update processes on other threads now that I understand that my assumptions were wrong, and it all makes so much more sense.

Thank you very much for both your time and patience.