OPNsense won't boot after upgrade to 25.1.8_1

[KeithRBrown]

Hi,

After upgrading from 25.1.7_4 to 25.1.8_1, OPNsense fails to boot.

The boot sequence hangs indefinitely after it displays "masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000".

However, if I go into boot options, and turn on verbose logging, it boots without any issue.

I also tried switching to kernel.old, but the boot sequence still hangs at the same point.

Thanks,

Keith

[Kewin]

Seems to be the same issue as I'm experiencing in https://forum.opnsense.org/index.php?topic=47635.0 ?

What hardware are you running on?

[KeithRBrown]

Hi Kewin,

I saw your post, which is why I tried kernel.old, but that didn't work for me. Although our issues do seem similar.

My hardware is an old PC, running an i5-8600K, booting from a SSD (formatted ZFS I believe), and a dual intel 10G NIC. I've been using this for over a year now, without any issues.

[Kewin]

Hi Keith

I can also get mine to boot with default kernel using the verbose mode 🤯

I've tried installing older kernels, to no avail.

Something about our setups must be similar, since it's only a few of us hitting this issue, and not all.. I've run the same hardware with OPNsense for 14 months without a single issue.. (Intel N100 (built in GPU), 16GB DDR5, NVMe SSD (formatted ZFS), 4x i226-V 2.5GbE. Using serial output for console).

And in my wisdom I've setup my box to auto-update.. and never considered using ZFS snapshots... so my only real way back into a stable state, is reinstalling the entire box using a configuration backup.. Which will probably be my next step..

[franco]

Microcode issues with the lastest release on their end perhaps?

[Kewin]

Microcode issues with the lastest release on their end perhaps?

Hey.. That WORKED!..

I uninstalled os-cpu-microcode-intel and rebooted.. And it came up without issues 🤯

Also upgraded to 25.1.9 and current kernel and it still boots without hanging now..

What let you on the microcode path?

Keith, can you reproduce?

[franco]

> Intel N100

;)

You might be able to use the former microcode version from Intel without issues, but I'm not sure at the moment which OPNsense carried the older one (probably 25.1.7 or 25.1.6).


Cheers,
Franco

[franco]

That would be 25.1.6 with the cpu-microcode-intel package:

# opnsense-revert -r 25.1.6 cpu-microcode-intel
# pkg lock -y cpu-microcode-intel

To me the whole idea of shipping these microcode updates by default and then Intel coming in and breaking boot of their hardware seemed like a good approach to not include it by default in the first place.  It's a bit weird if you ask me.  No new update so far...

commit c3583200d965ada0d6bec6425bdc18216eef8e53
Author: Franco Fichtner <franco@opnsense.org>
Date:   Tue May 13 11:22:24 2025 +0200

    sysutils/cpu-microcode-intel: sync with upstream


Cheers,
Franco

[KeithRBrown]

Uninstalling the microcode plugin also worked for me.

Any idea why booting in verbose mode works though? Seems a bit odd.

Is it best to leave the microcode plugin uninstalled for the moment?

[franco]

The old version should work as discussed.

It could be a bug in conjunction with the FreeBSD code and fixed there, but that would require some developer being interested and I still doubt Intel should cause bricking OS boots because of their microcode changes.

My assumption is this will disappear in the next microcode update not to be spoken off again.


Cheers,
Franco

[DespairServices]

Just wanted to chime in — I was getting sporadic full kernel crashes (trap 13 general protection faults) on boot with an i3-14100 and recent DDR5 setup. It wasn't consistent, but happened often enough to be a major issue.

After some digging, I traced it back to the os-cpu-microcode-intel plugin. Once I removed the plugin, the crashes completely stopped (at least so far).

Also worth noting: when I reinstalled the plugin just now to confirm, I got this message:

Message from x86info-1.31.s03_1:

===> NOTICE:

This port is deprecated; you may wish to reconsider installing it:

Abandoned upstream, fails to identify anything remotely new according to upstream issue reports.

It is scheduled to be removed on or after 2025-06-30.

===

Not sure if that was always there, but I suspect the included tooling might not play nice with newer 13th/14th-gen CPUs. If you're running modern Intel hardware and seeing weird instability, try removing the microcode plugin and see if it helps.

[meyergru]

Considering what is known about fried 14th gen Intel series now, it is well possible that Intel "fixed" something to further lower voltages or frequencies and botched stability during that the process.

Also, normally, CPU manufacturers intend to distribute their microcode updates to the mainboard and computer manufacturers, which can / should also adapt their BIOS settings accordingly. If you change the microcode only, it might break things.

So while you can patch security issues by applying microcode updates, you obviously can get more than you bargained for, when a CPU manufacturer "fixes" something else along they way...

[DespairServices]

Considering what is known about fried 14th gen Intel series now, it is well possible that Intel "fixed" something to further lower voltages or frequencies and botched stability during that the process.

Also, normally, CPU manufacturers intend to distribute their microcode updates to the mainboard and computer manufacturers, which can / should also adapt their BIOS settings accordingly. If you change the microcode only, it might break things.

So while you can patch security issues by applying microcode updates, you obviously can get more than you bargained for, when a CPU manufacturer "fixes" something else along they way...

Thanks — I had the impression that the "fried 14th-gen" issue mainly affected unlocked chips (i7/i9 K-series) under extreme overclocking or with unstable BIOS power delivery settings. I didn't think a locked i3 like mine would be impacted, especially since it's running at stock settings on a basic router board with no XMP or tuning options.

But I get your point — if Intel snuck in microcode changes to globally reduce voltage or clock margins, even locked chips could get hit by instability, especially if the BIOS wasn't updated to compensate.

[meyergru]

Exactly, and that is just because the chips (and the microcode updates) are basically the same. The good news is that it will probably not fry with either microcode version... ;-)