Second ZFS-Raid-Member doesn't boot after swap

[BusinessTux]

Hi there,

I have Scope-7 appliance with two ssd's. One of them is faulty, because my system didn't start after reboot this morning. After I removed the faulty one, OPNsense come online again.

So I swapped the ada1 device with a new one. This new ssd was not empty, but I backuped/restored the layout from ada0, like describe here.

Code Select Expand

root@OPNsense:~ # zpool attach zroot ada0p4 ada1p4
root@OPNsense:~ # zpool status
  pool: zroot
state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Feb 18 10:37:03 2023
        6.74G scanned at 363M/s, 1.13G issued at 60.9M/s, 6.74G total
        1.20G resilvered, 16.77% done, 00:01:34 to go
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada0p4  ONLINE       0     0     0
            ada1p4  ONLINE       0     0     0  (resilvering)

errors: No known data errors
root@OPNsense:~ # zpool status
  pool: zroot
state: ONLINE
  scan: resilvered 6.98G in 00:01:52 with 0 errors on Sat Feb 18 10:38:55 2023
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada0p4  ONLINE       0     0     0
            ada1p4  ONLINE       0     0     0

errors: No known data errors
root@OPNsense:~ # dd if=/dev/ada0p1 of=/dev/ada1p1
532480+0 records in
532480+0 records out
272629760 bytes transferred in 97.602079 secs (2793278 bytes/sec)
root@OPNsense:~ # dd if=/dev/ada0p2 of=/dev/ada1p2
1024+0 records in
1024+0 records out
524288 bytes transferred in 0.065542 secs (7999318 bytes/sec)
root@OPNsense:~ # zpool status
  pool: zroot
state: ONLINE
  scan: resilvered 6.98G in 00:01:52 with 0 errors on Sat Feb 18 10:38:55 2023
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada0p4  ONLINE       0     0     0
            ada1p4  ONLINE       0     0     0

errors: No known data errors
root@OPNsense:~ # gpart show
=>       40  468862048  ada0  GPT  (224G)
         40     532480     1  efi  (260M)
     532520       1024     2  freebsd-boot  (512K)
     533544        984        - free -  (492K)
     534528   16777216     3  freebsd-swap  (8.0G)
   17311744  451549184     4  freebsd-zfs  (215G)
  468860928       1160        - free -  (580K)

=>       40  488397088  ada1  GPT  (233G)
         40     532480     1  efi  (260M)
     532520       1024     2  freebsd-boot  (512K)
     533544        984        - free -  (492K)
     534528   16777216     3  freebsd-swap  (8.0G)
   17311744  451549184     4  freebsd-zfs  (215G)
  468860928   19536200        - free -  (9.3G)


I've copied ada0p1 for efi und ada0p2 for freebsd-boot via dd. But if I start the the applicance with only ada1 there is no bootable device.

How can I make ada1 bootable?

Thanks
Ulf