Can't install on a PCEngine APU4

[Astaoth]

Hi,

Since the 21.7 I'm unable to upgrade or install a new version of OPNSense on my APU 4. I thougth it was because of some changes on the HardenedBSD core, so I waited the version 22.1 for reinstalling my fw.

But with this version, I'm still unable to install it from a USB key with a serial image. After booting from the USB key, I have the usual log outputs until the network interfaces are turned on, then some errors about a missing GPT table at the end of the media and then nothing else happens.

However if I try to do the same with the version 21.1, it works fine. I still have the missing GPT header error, but the boot process continue and I can setup my fw.

Am I the only one with this problem ?

[szty0pa]

Hey,

I have in fact just upgraded an apu4 from 21.7.1 to 21.7.8 to 22.1 a few hours ago, it went smoothly. (I do have firewalling/routing issues on my production boxes under 22.1 but that's a different question.)

I also had GPT errors at one point, for me it was because of an older linux installation existed on the apu4 at first. (I think if you install OPNsense on the internal SSD, the GPT errors should go away.)

What firmware version ( https://pcengines.github.io/ ) do you have on the apu4? That could make a difference.

[Astaoth]

Thank you for the suggestion. I have the v4.0.33, maybe it's time for an upgrade
Which version would you recommend for the OPNSense compatibility and stability ?

[szty0pa]

No worries.
I don't have much experience yet, but the latest one (v4.15.0.2) seems to work fine, even with IOMMU (amd-v) enabled in BIOS despite the recommended and default setup. (It is needed for passing through the wifi to an OpenWRT vm running under OPNsense to have better hw support - though this should arrive in FreeBSD soon, so need for the OpenWRT vm.)

[Astaoth]

After a firmware upgrade to the 4.15.0.2, I still have the same problem 

[cookiemonster]

Upgraded mine from 21.7.5 to 21.7.8 followed by 22.1 but were in-place upgrades, with Boot Environments on ZFS. The last install from USB was to 21.7 so there is no reason it can't be done.
You have a couple of problems to eliminate I think.
First of all the usb media.
According to https://pcengines.github.io/ note the following note:
(2) - all apu boards have problems with USB 3.x stick detection in BIOS, system is working properly, but after reboot/warmboot/coldboot USB stick may not appear in BIOS boot menu (depends on the stick, well working sticks are mentioned in this document).
Some bios images have known issues, for instance 4.14.0.6:
Known issues:
    apuled driver doesn't work in FreeBSD. Check the GPIOs document for workaround.
    some PCIe cards are not detected on certain OSes and/or in certain mPCIe slots. Check the mPCIe modules document for solution/workaround.
    booting with 2 USB 3.x sticks plugged in apu4 sometimes results in detecting only 1 stick
    certain USB 3.x sticks happen to not appear in boot menu
    booting Xen is unstable
So you need to see your installation method aligns with these known issues,
I am using that bios version without problem. I used a non-USB3 stick for the installer and installed to an internal mSSD.

Next is what media are you trying to install to? That would be useful information. Is that healthy?
You could looking at your partitions and see if you're missing the backup copy because "some errors about a missing GPT table at the end of the media ". However if you're not comfortable using gpart (see for example http://www.wonkity.com/~wblock/docs/html/disksetup.html) then my suggestion is to get another media so you can go back to your current if needed, replace it and install from scratch, followed by restore a config backup.

[Astaoth]

Hi, thank you for your answer

I'm aware of the PCEngine issues with the USB sticks. With the one I use, I have been able to setup older OPNSenses, until the 21.1, with no problems. I don't have anymore any USB 2 stick.

I would like to install it to an internal mSSD device, which is healthy. However, I'm stuck during the boot process, right after the network interfaces are turned on, I don't have access to the setup.

About the missing GPT backup table, I had this errors as well with other OPNSense installs, but this didn't prevent me to make them.
I have thought of doing the setup on the internal drive on an other endpoint, but instead I would have rather prefered to solve this issue. But since it seems I'm the only one with this problem, I'll do it anyway, when I'll have some freetime, with the hope than someone has an other idea to test in the meantime

[cookiemonster]

With it working for others it leaves you with having to use an elimination process.
It could be the USB stick or even the port. My thinking is hardware, not something broken but not setup as it should.
For this you could:
- try to use the internal usb headers (needs a cable not included but you might have lying around)
- try to boot from an SD card
- wipe and re-format the usb stick (I suspect this where the problem is)
- connect it via serial to another freebsd computer using cu -- call   UNIX (this one is not so simple)
- burn a vanilla freebsd and use the opn convert script once/if it boots correctly from the same usb stick
- what you say, do the installation to mSSD on another machine.
Good luck.

[MenschAergereDichNicht]

Initially i also had problems installing 22.1 on a APU4. In my case it worked after i switched to a USB 2 stick. I am not sure if it is a problem with 22.1 or the BIOS version. But when running in USB 3 mode i could not succesfully boot from the stick.

Bios version 4.14.0.6 works for me with 22.1 after solving the boot problem.

[fabianodelg]

This worked for me too on an APU4. Great trick!