Does ZFS support 16gb memory cards?

[SkyHiRider]

On my apu4, I've booted from a usb stick and installed the latest version of OPNsense onto my 16gb memory stick, but boot fails when selecting it.

I am still investigating, but wondering if ZFS does not support SD memory cards, or if there are other requirements?

I am looking into OPNsense after using PFsense for years. Hoping to get over the initial setup pains to get a full system working during Christmas.

I've already updated my bios to 4.17.0.3, went well via the live image.

[doktornotor]

ZFS is not involved at that stage. You need serial on nano image and MBR install.

[SkyHiRider]

Thank you, I downloaded the serial image, not realizing I needed another one for SD cards.

[Patrick M. Hausen]

Thank you, I downloaded the serial image, not realizing I needed another one for SD cards.

You need a serial image for an APU device which only has got a serial console. This is independent of the storage medium. My APUs have an M.SATA SSD - they also use the serial image.

[SkyHiRider]

Thank you, I downloaded the serial image, not realizing I needed another one for SD cards.

You need a serial image for an APU device which only has got a serial console. This is independent of the storage medium. My APUs have an M.SATA SSD - they also use the serial image.

Haven't tested the nano image yet, but its description states "nano: a preinstalled serial image for USB sticks, SD or CF cards as MBR boot. These images are 3G in size and automatically adapt to the installed media size after first boot."

So the nano is serial by default, while the other "serial" option seems to be aimed at mSata drives, not sd cards.

I missed it, thinking serial was enough when reading through the first three options.

[jjelliott]

For installation on an SD card, I think it may be better to use the nano version - it's an image file you deploy directly onto an SD card (using any one of several disk image writers like rufus (on Windows)).   It uses UFS (not ZFS), so I'm not sure this answers the OP's question.

If the OP wants to experiment with ZFS on an SD card, then yes, he should use the serial installer, and try installing to the SD card from that.  (I have installed pfSense onto an SD card using their serial installer, using UFS and no swap to minimize disk writes.  It worked fine.  I did not try ZFS on the SD card.) 

One question about ZFS - does it do more writes to the disk than UFS, and would it wear the SD card out faster?

[SkyHiRider]

Well, I put the nano image onto a usb an tried both ZFS and UFS, and both result in an unbootable media:

Code: [Select]

Booting from Hard Disk...
Boot failed: could not read the boot disk

Any recommendations what could be causing this? As far as I know the memory card is ok, and I am using the correct image.

When I was installing pfSense a few years ago I never had these issues, would like to actually get to the part where I configure the router.

[jjelliott]

The nano image is UFS.  When you write the image onto an SD card, it is written as UFS.  As far as I know, ZFS isn't an option with the nano image.

I tried installing with ZFS onto an SD card using the serial install image.  It is working great.

To reduce unnecessary writes to the card, during install I set swap to 0 GB (a trick that was mentioned by one of the Netgate developers).  I also minimize logging and set /var and /tmp to run in RAM. 

I have been watching the writes to disk by SSH'ing into the console and issuing 'iostat -x'.  As I type this, kr/s (kB written per second since boot) sits at 6.1.  (It started high right after boot and has been dropping since.)  Assuming a reasonable sized SD card (say 32 GB) of high quality (Samsung Pro Endurance uses good MLC flash chips), I've calculated that the card should last many many years (decades).

[Edit - mistyped the kr/s - corrected to 6.1.  It keeps coming down over time, indicating there's little writing going on once the system is booted and running.]

[SkyHiRider]

Have you tried installing from USB to the SD card?

Thats what I've been doing - put installation image (nano or serial) to usb, plug usb into apu4, boot from said usb and install to memory stick.

Installation goes well, but fails after rebooting as it could not read the boot disk.

[doktornotor]

The nano image is a raw image. You do not install it. You boot it and configure.

https://docs.opnsense.org/manual/install.html#embedded-vs-full

[SkyHiRider]

The nano image is a raw image. You do not install it. You boot it and configure.

https://docs.opnsense.org/manual/install.html#embedded-vs-full

Thanks, burning the nano image onto another SD card directly and booting from that worked.

I just find it odd that the previous card failed botting after serial installation every time, and this one failed during installation when attempting to create partitions.

[jjelliott]

Have you tried installing from USB to the SD card?

Yes, that's what I did to get ZFS on the SD card.  SD card in the slot on the APU motherboard, serial installer USB stick in one of the rear panel USB ports.  Boot from the USB stick, install onto the SD card.

Glad doktornotor's feedback helped you get started.  Just remember, the nano image uses UFS on the SD card.  If you want ZFS, you'll need to boot from the serial installer USB stick and install from there onto your SD card.