I am looking to add a second SSD.
Upon first install, I installed using ZFS on a single SSD.
I recently acquired another SSD and I wanted to mirror it. I know it's an option on first install.
I am looking to avoid needing to backup, and fresh install.
How can I add it in, without fresh installing?
Assuming your first SSD is ada0 and your second ada1. Further assuming your partition table looks like this (apart from the sizes, probably):
root@opnsense:~ # gpart show ada0
=> 40 537234688 ada0 GPT (256G)
40 409600 1 efi (200M)
409640 1024 2 freebsd-boot (512K)
410664 984 - free - (492K)
411648 16777216 3 freebsd-swap (8.0G)
17188864 520044544 4 freebsd-zfs (248G)
537233408 1320 - free - (660K)
You can go about the task like this:
# copy partition table
gpart backup ada0 | gpart restore -F ada1
# copy EFI partition
dd if=/dev/ada0p1 of=/dev/ada1p1
# copy legacy boot loader partition
dd if=/dev/ada0p2 of=/dev/ada1p2
# attach second disk to ZFS pool
zpool attach zroot ada0p4 ada1p4
# turn swap partition into mirrored device
gmirror load
swapoff -a
gmirror label -b round-robin swap ada0p3
gmirror configure -a swap
gmirror insert swap ada1p3
Change the file /etc/fstab to look like this:
# Device Mountpoint FStype Options Dump Pass#
#
/dev/mirror/swap none swap sw 0 0
Re-enable swap:
swapon -a
HTH
Patrick
Many thanks for the instructions, Patrick.
I did add a second SSD, but in my case ada1 is the original one and ada0 is the second one:
root@hush:/home/pippog # gpart show
=> 40 250069600 ada1 GPT (119G)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 16777216 3 freebsd-swap (8.0G)
17311744 232757248 4 freebsd-zfs (111G)
250068992 648 - free - (324K)
and
root@hush:/home/pippog # gpart show ada0
gpart: No such geom: ada0.
So, all I have to do is to invert ada0 and ada1 in Patrick's instructions, right?
If so, are those correct?
1. Assuming your first SSD is ada1 and your second ada0:
# copy partition table
gpart backup ada1 | gpart restore -F ada0
# copy EFI partition
dd if=/dev/ada1p1 of=/dev/ada0p1
# copy legacy boot loader partition
dd if=/dev/ada1p2 of=/dev/ada0p2
# attach second disk to ZFS pool
zpool attach zroot ada1p4 ada0p4
# turn swap partition into mirrored device
gmirror load
swapoff -a
gmirror label -b round-robin swap ada1p3
gmirror configure -a swap
gmirror insert swap ada0p3
2. Change the file /etc/fstab to look like this:
# Device Mountpoint FStype Options Dump Pass#
#
/dev/mirror/swap none swap sw 0 0
3. Re-enable swap:
swapon -a
Tia.
Looks good. Make it so ;)
Hey so just getting around to trying this.
Sounds good, my new drive is also going to be ada0.
However when running
gpart backup ada1 | gpart restore -F ada0
I run into a gpart: size '983025664' invalid argument error. I assume this is because the new drive is smaller in size, so the partition sizes won't match.
Can you please help clarify how to safely proceed. Thanks.
Here is the disk layout
# geom disk list
Geom name: ada0 (new second disk adding in)
Providers:
1. Name: ada0
Mediasize: 250059350016 (233G)
Sectorsize: 512
Mode: r0w0e0
descr: SanDisk SDSSDH3 250G
lunid: 5001b448b243e95c
ident: 230206A0004C
rotationrate: 0
fwsectors: 63
fwheads: 16
Geom name: ada1 (original disk)
Providers:
1. Name: ada1
Mediasize: 512110190592 (477G)
Sectorsize: 512
Mode: r1w1e2
descr: SanDisk SDSSDH3 512G
lunid: 5001b448ba4679ef
ident: 21120U801225
rotationrate: 0
fwsectors: 63
fwheads: 16
..also for clarification I will never come close to even using 100gb (nevermind the 250-512gb which the drives are) so size should not be an issue.
I think I'm only using 2.2GB/452 on my current drive haha
Short answer: You cannot add a smaller drive. Period. Do a fresh install. Then reimport your saved configuration.
Longer answer: You cannot shrink a ZFS vdev. Fundamentally impossible. So you can only ever attach equal size or larger disks to an existing vdev. You can do a mirrored install to a set of differently sized disks, but the resulting size will be that of the smaller disk. Since you already have a running installation the size of the larger disk, the only way is a clean reinstall.
Thank you for this guide. The first steps work well for me, and then this:
root@OPNsense:~ # gmirror load
gmirror: Command 'load' not available; try 'load' first.
I searched this error and got a few hits. One was way over my head and the others didn't offer anything that appeared to help my situation.
This isn't a show-stopper for me, as it's just the swap partitions that are failing to mirror, but I'm still interested in any insights people may have on this error.
Edit: OPNsense 23.1-amd64
I haven't run updates since installing 23.1 fresh.
Possibly it is already loaded? What is the result of "kldstat"?
So, with great courage I went ahead, restarted the firewall and yes, it worked 8)
How do I check the mirroring is up and runing? :P
And is there way for newbies to monitor the mirror ?
zpool status
gmirror status
No dashboard widget yet, as far as I know.
Sorry, can you please confirm if it looks fine?
Tia.
Yes, it does. It will tell you with words quite different from anything like "complete" or "online" if anything breaks.