[HOWTO] Shrink your swap & expand your ZFS root partition

[mbentley]

Problem: if you have a small disk, the default swap partition size can be excessively large compared to the ZFS root which can be a big problem for keeping logs or upgrading. In my case, I am using a 16 GB Intel Optane NVMe drive due to it's write durability and they're pretty cheap on ebay.

I re-created this in a VM with a 10 GB disk to show what I did.  Here is the current disk layout:

Code Select Expand

# gpart show
=>      40  20971440  da0  GPT  (10G)
        40    532480    1  efi  (260M)
    532520      1024    2  freebsd-boot  (512K)
    533544      984      - free -  (492K)
    534528  16777216    3  freebsd-swap  (8.0G)
  17311744  3657728    4  freebsd-zfs  (1.7G)
  20969472      2008      - free -  (1.0M)

You can see that in order to expand the zfs partition, the swap needs to be re-created and the zfs partition shifted toward the beginning of the disk.

In order to do the move of the ZFS root partition, I used a gparted live cd.  In gparted:

Note: You may get a message about not all of the space on the GPT partition being used.  You can either hit ignore multiple times or hit fix.  I've done it both ways and it doesn't seem to impact OPNsense regardless.

• Delete the 8GB swap partition (p3)
• Move the ZFS partition (p4) left but in the "Free space preceding" box, add a 1024M (or whatever size you want your new swap to be) gap to re-create the swap partition from OPNsense
• Apply changes

Once this process finishes, reboot back into OPNsense and connect to the console or via SSH.

1. Check the updated partition layout - you should have a gap between boot and ZFS for your swap to be re-created and free space after your ZFS partition to allow for expansion:

Code Select Expand

# gpart show
=>      40  20971447  da0  GPT  (10G)
        40    532480    1  efi  (260M)
    532520      1024    2  freebsd-boot  (512K)
    533544   2098136       - free -  (1.0G)
   2631680   3657728    4  freebsd-zfs  (1.7G)
   6289408  14682079       - free -  (7.0G)
2. Expand the zfs partition to fill the remaining space:

Code Select Expand

gpart resize -i 4 da0
3. Check that the zfs partition has been expanded:

Code Select Expand

# gpart show
=>      40  20971447  da0  GPT  (10G)
        40    532480    1  efi  (260M)
    532520      1024    2  freebsd-boot  (512K)
    533544   2098136       - free -  (1.0G)
   2631680  18337792    4  freebsd-zfs  (8.7G)
  20969472      2015       - free -  (1.0M)

4. Get the current size of the zpool:

Code Select Expand

# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zroot  1.62G  1.47G   155M        -         -    54%    90%  1.00x    ONLINE  -

5. Expand the zpool:

Code Select Expand

zpool online -e zroot da0p4

6. Check the new size of the pool:

Code Select Expand

# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zroot  8.62G  1.47G  7.15G        -         -     8%    17%  1.00x    ONLINE  -

7. Find the beginning of the free space before the zfs partition (in this case, it's 533544):

Code Select Expand

# gpart show
=>      40  20971447  da0  GPT  (10G)
        40    532480    1  efi  (260M)
    532520      1024    2  freebsd-boot  (512K)
    533544   2098136       - free -  (1.0G)
   2631680  18337792    4  freebsd-zfs  (8.7G)
  20969472      2015       - free -  (1.0M)

8. Re-create the swap starting at the beginning of the free space:

Code Select Expand

gpart add -t freebsd-swap -b 533544 -l swap0 da0

9. Make sure the partition shows as expected (there may or may not be some small amount of free space between the boot and swap):

Code Select Expand

# gpart show
=>      40  20971447  da0  GPT  (10G)
        40    532480    1  efi  (260M)
    532520      1024    2  freebsd-boot  (512K)
    533544       984       - free -  (492K)
    534528   2097152    3  freebsd-swap  (1.0G)
   2631680  18337792    4  freebsd-zfs  (8.7G)
  20969472      2015       - free -  (1.0M)

10. Check the fstab entry to ensure it matches as expected (it should):

Code Select Expand

# cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/gpt/efiboot0               /boot/efi       msdosfs rw              2       2
/dev/da0p3              none    swap    sw              0       0

11. Turn swap on using the device name in fstab:

Code Select Expand

swapon /dev/da0p3

12. Verify swap is on:

Code Select Expand

# swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/da0p3        1048576        0  1048576     0%