Applied latest updates for 25.1 and now system won't boot

[7queue]

On a system that has been running just fine with 24.7 and 25.1 prior to the latest update.

I applied the latest updates and it throws an error when booting now:

CPU0:<ACPI CPU> on acpi0

The system uses DDR5 so it's relatively new hardware.

I reinstalled 25.1 and applied the update with the same result.

Also tried installing the latest FreeBSD on this system and applied updates and it boots fine.

Any hint at how I could track this issue down?

I'm going to try installing 25.1 and apply the updates on a different system and see what happens.

8  )

[EricPerl]

A picture of the screen might contain some clues...
You have disclosed so little information.

[7queue]

It seems specific to the particular system it was working on.

If I lock the base package and update it boots so it is something in the base package that has changed.

Using verbose boot it errors and stops at trying to initializing the uart on the motherboard, no actual serial port on the board but a uart header. I haven't tried attaching a serial adapter to the uart yet since I don't remember where I put them.

Right now I'm reading up on how to clone the repo and build the project to then try and find the base package and look through the code.

Thanks 8  )

[7queue]

I'm leaning towards this: After some back and forth today we are rolling back a console default change done in FreeBSD 14.2 that we do not think is necessary at this particular point in time.

In FreeBSD 14.2, the default console behavior is primarily driven by the vt(4) driver, which is now the recommended console driver, replacing the older syscons(4).

[7queue]

Figured it out.

After the update when it reboots hit <ESC> at the boot screen then enter
# set hint.uart.0.disabled=1
# set hint.uart.1.disabled=1
# boot

After it boots login and edit /boot/device.hints and change it like so then reboot to test the changes:

hint.atkbdc.0.at="isa"
hint.atkbdc.0.port="0x060"
hint.atkbd.0.at="atkbdc"
hint.atkbd.0.irq="1"
hint.psm.0.at="atkbdc"
hint.psm.0.irq="12"
hint.sc.0.at="isa"
hint.sc.0.flags="0x100"
#hint.uart.0.at="isa"
#hint.uart.0.port="0x3F8"
#hint.uart.0.flags="0x10"
#hint.uart.0.irq="4"
#hint.uart.1.at="isa"
#hint.uart.1.port="0x2F8"
#hint.uart.1.irq="3"
hint.atrtc.0.at="isa"
hint.atrtc.0.port="0x70"
hint.atrtc.0.irq="8"
hint.attimer.0.at="isa"
hint.attimer.0.port="0x40"
hint.attimer.0.irq="0"
hint.acpi_throttle.0.disabled="1"
hint.p4tcc.0.disabled="1"
hint.uart.0.disabled="1"
hint.uart.1.disabled="1"

This worked for me and I learned a lot on this adventure!

8  )

[EricPerl]

...
Using verbose boot it errors and stops at trying to initializing the uart on the motherboard, no actual serial port on the board but a uart header. I haven't tried attaching a serial adapter to the uart yet since I don't remember where I put them.
...

That's the kind of clue I was after. There have been several threads about these uart settings over the past few weeks.

[pippcz]

Figured it out.

After the update when it reboots hit <ESC> at the boot screen then enter
# set hint.uart.0.disabled=1
# set hint.uart.1.disabled=1
# boot

After it boots login and edit /boot/device.hints and change it like so then reboot to test the changes:

hint.atkbdc.0.at="isa"
hint.atkbdc.0.port="0x060"
hint.atkbd.0.at="atkbdc"
hint.atkbd.0.irq="1"
hint.psm.0.at="atkbdc"
hint.psm.0.irq="12"
hint.sc.0.at="isa"
hint.sc.0.flags="0x100"
#hint.uart.0.at="isa"
#hint.uart.0.port="0x3F8"
#hint.uart.0.flags="0x10"
#hint.uart.0.irq="4"
#hint.uart.1.at="isa"
#hint.uart.1.port="0x2F8"
#hint.uart.1.irq="3"
hint.atrtc.0.at="isa"
hint.atrtc.0.port="0x70"
hint.atrtc.0.irq="8"
hint.attimer.0.at="isa"
hint.attimer.0.port="0x40"
hint.attimer.0.irq="0"
hint.acpi_throttle.0.disabled="1"
hint.p4tcc.0.disabled="1"
hint.uart.0.disabled="1"
hint.uart.1.disabled="1"

This worked for me and I learned a lot on this adventure!

8  )

Thank you man! This saved my ass today after I ran the update, rebooted and got stuck... Is there a plan to address the UART settings issue at scale or is everyone affected expected to fix it on their own? Asking as I have another firewall that is remote and pending the same set of updates. I can configure the "/boot/device.hints" file as suggested above. But will it boot 100%? Is the remote unit also affected? Is there a way to confirm it before the reboot? Thanks!

[roylaprattep]

Your best bet, do a snapshot before upgrading next time.

[Patrick M. Hausen]

This UART issue only affects really old systems. Architecture-wise, not referring to the date of purchase. Prime example: APU(2|4). These are *ancient* ;-)

[OPNenthu]

It also affects some current production Protectli units which are running coreboot.  I was told by a rep that they are working with their coreboot vendor on a fix but in the meantime "hint.uart.0.at=isa" is needed in system tunables.

[franco]

This was fixed in 25.1.6 BTW.

[OPNenthu]

Nice!  Serial console is working again here after deleting the tunable.

Bonus: I'm able to authenticate again in PuTTY by copy/pasting my login password.  That had stopped working with the tunable and I was having to manually type the password in.  No clue why that makes a difference in PuTTY but it was irritating :)

[pippcz]

Your best bet, do a snapshot before upgrading next time.

How exactly is a snapshot going to help me with a physical box in a remote site? I would still need to go there if the system doesn't boot - which is this exact situation. If there was a way to automatically fall-back to old-good snapshot when default fails, that would be great. But when the system gets stuck during the boot and hard reboot is the only way to get another try...

This UART issue only affects really old systems. Architecture-wise, not referring to the date of purchase. Prime example: APU(2|4). These are *ancient* ;-)

It's an Intel N5105 based system and I am definitely the only one running this HW. I would even guess that the population of OPNsense users running similarly "ancient" model isn't small at all. Anyways this is really up to the OPNsense devs how they want to build the trust with their users. Breaking remote sites is not fun.

This was fixed in 25.1.6 BTW.

What exactly was fixed in 25.1.6? I updated to 25.1.8 yesterday and the system didn't boot because of the UART issue. Was your comment related to Protectli units only or the UART issue generally? Thanks!

[franco]

I think you are misleading us with your posts and/or you are looking at the wrong cause for your issue (recent Intel microcode is also breaking some systems).

The UART not working for older hardware *was* fixed in 25.1.6. The age of the hardware can be irrelevant as the BIOS is responsible for advertising a device as ISA or ACPI in FreeBSD. FreeBSD *thought* that ISA is deprecated, but their code is likely faulty because the consoles are not even connected via ISA in the first place.

https://github.com/opnsense/src/commit/52bb0884e6

That being said they did try to fix newer devices not booting  so now these devices don't have a console anymore.

If there was a best of both worlds we would go for it, but going back to the old way that works for most is the best approach at the moment.

[franco]

If it helps here is the origin of the FreeBSD 14.2 UART default fiddling... https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270707