upgrade to 25.7 -> bootloader too old

[Patrick M. Hausen]

First: don't reboot :-)

Second: the startup.nsh is not necessary

Third: the system will boot just fine with just /boot/efi/efi/boot/bootx64.efi

So I would:

Code Select Expand

rm -r /boot/efi/efi
mkdir -p /boot/efi/efi/boot
cp /boot/loader.efi /boot/efi/efi/boot/bootx64.efi

Then check if there is enough free space to also create and populate the FreeBSD subdirectory. But it's not strictly necessary.


The reason of all this is historical. The concept behind EFI is to have a sufficiently large FAT32 partition and in there a structure roughly like this:

efi/freebsd/loader.efi
efi/windows/something.efi
efi/debian/something.efi
...

and then to be able to pick the OS at boot time or at least set a fixed OS in the BIOS setup.

And then there is a fallback path that is according to the standard only to be used on removable media - e.g. DVSs or USB drives:

efi/boot/bootx64.efi

or

efi/boot/aarch64.efi - if you are running on 64 bit ARM.

But it's supposed to be only a fallback.

FreeBSD when introducing EFI support got it wrong and

- installed to efi/boot/bootx64.efi
- made the EFI partition way to small

Which leads to the situation you have now.

So, delete everything, create the efi/boot/bootx64.efi path only - your system should (!) boot fine after that.


Plan some time for a reinstallation with ZFS and a reasonably large EFI partition ;-)

HTH,
Patrick

[tessus]

Ok, I have removed the /boot/efi/efi/freebsd again. At least the file fit in /boot/efi/efi/boot

Plan some time for a reinstallation with ZFS and a reasonably large EFI partition ;-)

I suppose the sizing will be done by the OPNsense installer, will it not?

I have always thought that migrating to ZFS works like this:
- Save config
- Reinstall (which wipes out everything) using the OPNsense.iso on a USB stick
- Restore config

Thus I also thought that the installer would create the layout for my disk. Well, even if it doesn't, I will certainly make the bootfs bigger than 512KB. It's an overkill, but I will go for 5 or 10MB.

The more interesting question is, whether I can restore a 25.1.12 config on a 25.7.x system. I suspect it is possible, but it doesn't hurt to ask. In that case, I will do a fresh install of 25.7 when I find the time to hook up a monitor and keyboard.

[Slashing]

@Patrick Sure, here you go:

Code Select Expand

root@cator00r:~ # gpart show -l
=>       40  234441568  ada0  GPT  (112G)
         40     409600     1  (null)  (200M)
     409640       1024     2  bootfs  (512K)
     410664  215567272     3  rootfs  (103G)
  215977936   16777216     4  swapfs  (8.0G)
  232755152    1686456        - free -  (823M)

Maybe first correct the file system type from null to efi? For example - "gpart modify -i 1 -t efi ada0"

[Patrick M. Hausen]

I suppose the sizing will be done by the OPNsense installer, will it not?

Yes, absolutely. It will completely wipe the disk and create a new layout. Including updated partition labels and fstab.

And yes, a 25.1 config can be imported to a 25.7 system.

[Patrick M. Hausen]

Maybe first correct the file system type from null to efi? For example - "gpart modify -i 1 -t efi ada0"

Yes. Thanks for noticing and posting.

The entire layout seems to be an artifact of an outdated FreeBSD installer. Not OPNsense's fault in particular. Best fixed by reinstalling.

[tessus]

@Patrick Sure, here you go:

Code Select Expand

root@cator00r:~ # gpart show -l
=>       40  234441568  ada0  GPT  (112G)
         40     409600     1  (null)  (200M)
     409640       1024     2  bootfs  (512K)
     410664  215567272     3  rootfs  (103G)
  215977936   16777216     4  swapfs  (8.0G)
  232755152    1686456        - free -  (823M)

Maybe first correct the file system type from null to efi? For example - "gpart modify -i 1 -t efi ada0"

Now I am getting really confused:

Code Select Expand

root@cator00r:~ # gpart show
=>       40  234441568  ada0  GPT  (112G)
         40     409600     1  efi  (200M)
     409640       1024     2  freebsd-boot  (512K)
     410664  215567272     3  freebsd-ufs  (103G)
  215977936   16777216     4  freebsd-swap  (8.0G)
  232755152    1686456        - free -  (823M)

root@cator00r:~ # gpart show -l
=>       40  234441568  ada0  GPT  (112G)
         40     409600     1  (null)  (200M)
     409640       1024     2  bootfs  (512K)
     410664  215567272     3  rootfs  (103G)
  215977936   16777216     4  swapfs  (8.0G)
  232755152    1686456        - free -  (823M)
So one shows EFI, and with the -l option it shows (null)

Then you asked me to put in fstab /dev/ada0p1 with mount point /boot/efi
ada0p1 is the 2nd partition (bootfs) and not the 200MB EFI

At least on Linux partitions are always 0-indexed, I suppose it is the same on FreeBSD. gpart just shows them 1-indexed.

So now I am rather confused.

P.S.: confused why they are shown s EFI and (null), not about mounting the bootfs (that was only because you mentioned to size the EFI partition differently, even though you probably meant the bootfs)

Yes, absolutely. It will completely wipe the disk and create a new layout. Including updated partition labels and fstab.

And yes, a 25.1 config can be imported to a 25.7 system.

Thanks, very nice. That should make things a lot easier.

[Patrick M. Hausen]

(null) is the label - there is none. That's the output of "gpart show -l" as opposed to just "gpart show". Sorry, I was equally confused. Your partition type (efi) is ok.

ada0p1 is the first partition (efi). ada0p2 would be the second one (freebsd-boot). See the output of "gpart show". The partitions start at 1, not at 0.

[tessus]

Thanks again for your reply. One last question (or actually 2). For me there are 2 OPMsense images relevant: vga (I used this type in 2021 when I installed OPNsense) and dvd.

Both fit on an USB stick so I don't see why there are 2 types. The explanation also seems to be the same:

vga: USB installer image with live system capabilities running in VGA mode as GPT boot. On amd64, UEFI boot is supported as well.
dvd: ISO installer image with live system capabilities running in VGA mode. On amd64, UEFI boot is supported as well.

What's the actual difference and which one should I use?

[Patrick M. Hausen]

For USB: vga
For burning to a DVD: dvd

[tessus]

See, this is what I don't understand. You can flash a DVD iso to an USB stick as well. And it boots. So I don't understand the difference.

Anyway, I'll just use the vga one. Thanks for all your help.

[Patrick M. Hausen]

Not necessarily - some systems treat USB drives as a hard disk and won't boot an ISO file system from them. The other way round a DVD must contain an ISO image and not a generic hard disk one.

[tessus]

Not necessarily - some systems treat USB drives as a hard disk and won't boot an ISO file system from them.

This is interesting. I've never run into this issue, ever. Luck, I guess.

a DVD must contain an ISO image and not a generic hard disk one.

Yep, that I knew.

[Patrick M. Hausen]

Anyway I hope you are successful in reinstalling/upgrading your system. Unfortunately there's a lot of history and technical debt involved in this whole mess.

I cannot speak for OPNsense but I guess their stance is "we are an appliance, reinstall and import your configuration and you will be fine."

The FreeBSD project has some work to do to come to a general solution. E.g. how OPNsense installs the boot partitions - always both - is not how I do it with short of 100 servers in my data centre. They have - depending on the platform - only a legacy or an EFI partition. That of course changes partition numbers compared to OPNsense so a "fool proof" runbook to upgrade your boot loader is a real challenge.

I will look into that script linked into this thread and take it to the FreeBSD developer summit in September.

Kind regards,
Patrick

[tessus]

I cannot speak for OPNsense but I guess their stance is "we are an appliance, reinstall and import your configuration and you will be fine."

I am ok with that, in a sense (pun intended). This is a valid argument and the only issue I have is that my OPNsense is headless. Connecting stuff to it is a hassle. I usually like to avoid it. So I am happy, if I just, upgrade, upgrade, upgrade. But at one point "old" issues catch up (my very small bootfs, old BIOS, old bootloader) and there is no other choice than doing a fresh install. And of course it probably wouldn't hurt to use ZFS instead of UFS.

Either way, OPNsense upgrades always went well and I am very impressed that all the major upgrades didn't mess up my firewall and always booted up again. (I read in this forum that people often had to reinstall after a major upgrade. This never happened to me.)

[franco]

I investigated a little bit and couldn't find a description of what constitutes a "too old" boot loader in a stable FreeBSD series from 14.2 to 14.3. I also do not remember this happening ever before.

Not that I'd favour starting to try and upgrade the boot loaders now, but the tiniest amount of precision from upstream on the matter would be something that should be in the actual release documentation to make that a possibility?


Cheers,
Franco