Call for testers - new CPU microcode update plugins

[Patrick M. Hausen]

UPDATED! Please check the list of packages needing a manual install if you do not notice any ucode updates happening.

Hi all,

apparently some (not a majority, but enough for this to be a problem) users are hit by a race condition in the FreeBSD kernel which leads to the system locking up at booting when the microcode update plugin is installed. The specific point here is that the ucode updates are loaded "early" by the standard plugin which means before regular kernel startup.

The FreeBSD package alternatively allows for "late" loading of the ucode updates. This is considered non-optimal, because CPU features might change after the kernel is started and detection of said features has happened.
Yet personally for more than 8 years since ucode updates became a thing I have never seen a problem with late loading. It simply means the kernel boot process and startup of some services will happen with an unpatched CPU, then the updates will be applied with the system up and running.

Specifically the people who experience the boot time lockups with the plugin installed - if you can afford some downtime and a couple of reboots, please try this and report back.

1. Uninstall the os-cpu-microcode-amd from the UI. You probably did this already, if you are running 26.7 and you were hit by this problem.

2. For Intel systems:

Code Select Expand

pkg install x86info
pkg install cpu-microcode-intel
pkg install cpu-microcode-rc
pkg add -f https://pkg.opnsense.org/FreeBSD:15:amd64/snapshots/misc/os-cpu-microcode-intel-devel-1.1_1.pkg

For AMD systems:

Code Select Expand

pkg install x86info
pkg install cpu-microcode-amd
pkg install cpu-microcode-rc
pkg add -f https://pkg.opnsense.org/FreeBSD:15:amd64/snapshots/misc/os-cpu-microcode-amd-devel-1.1_1.pkg

3. Reboot your system. You should be able to observe the late application of the ucode updates in /var/run/dmesg.boot after the reboot.

Thanks and kind regards,
Patrick

[Igor]

Done:

Code Select Expand

--<<BOOT>>---
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.1-RELEASE-p1 stable/26.7-n283674-12334a596709 SMP amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
[1] VT(efifb): resolution 800x600
[1] CPU microcode: updated from 0x10 to 0x21
[1] CPU: Intel(R) N100 (806.40-MHz K8-class CPU)
[1]  Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
[1]  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
--
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.1-RELEASE-p1 stable/26.7-n283674-12334a596709 SMP amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
[1] VT(efifb): resolution 800x600
[1] WARNING: sysctl vfs.zfs.arc_max is deprecated. Use vfs.zfs.arc.max instead.
[1] CPU microcode: updated from 0x10 to 0x21
[1] CPU: Intel(R) N100 (806.40-MHz K8-class CPU)
[1]  Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
[1]  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>


[db9]

I followed the pkg steps and rebooted the system. No microcode messages in /var/run/dmesg.boot after boot after seconds or minutes.

=====
Message from x86info-1.31.s03_1:
===>   NOTICE:

This port is deprecated; you may wish to reconsider installing it:

Abandoned upstream, fails to identify anything remotely new according to upstrea                                                                                     m issue reports.

It is scheduled to be removed on or after 2025-06-30.

=====
Message from cpu-microcode-intel-20260512_1:

--
To enable microcode updates at boot time:
cat << EOF >> /boot/loader.conf
cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/intel-ucode.bin"
EOF


-----

A bit early to ask, but what will be the step to have this converted to the final production version of the package?

[LOTRouter]

I followed the steps and also do not see the microcode load.  Under SYSTEM > FIRMWARE > PLUGINS I see "os-cpu-microcode-intel-devel (misconfigured)" with the Repository as "unknown-repository" so I believe I have it correctly loaded, but it doesn't appear to ever update the microcode:

Code Select Expand

---<<BOOT>>---
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.1-RELEASE-p1 stable/26.7-n283674-12334a596709 SMP amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
[1] VT(vga): resolution 640x480
[1] CPU: Intel(R) N100 (806.40-MHz K8-class CPU)
[1]   Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
[1]   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
[1]   Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
[1]   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
[1]   AMD Features2=0x121<LAHF,ABM,Prefetch>
[1]   Structured Extended Features=0x239ca7eb<FSGSBASE,TSCADJ,BMI1,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,NFPUSG,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
[1]   Structured Extended Features2=0x98c007bc<UMIP,PKU,OSPKE,WAITPKG,GFNI,VAES,VPCLMULQDQ,RDPID,MOVDIRI,MOVDIR64B>
[1]   Structured Extended Features3=0xfc184410<FSRM,MD_CLEAR,IBT,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
[1]   Structured Extended Features4=0x810
[1]   XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
[1]   IA32_ARCH_CAPS=0x180fd6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO,TAA_NO>
[1]   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
[1]   TSC: P-state invariant, performance statistics
[1] real memory  = 34358689792 (32767 MB)
[1] avail memory = 33219031040 (31680 MB)
[1] Event timer "LAPIC" quality 600
[1] ACPI APIC Table: <ALASKA A M I >
[1] WARNING: L3 data cache covers more APIC IDs than a package (7 > 3)
[1] FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
[1] FreeBSD/SMP: 1 package(s) x 4 core(s)
[1] random: registering fast source Intel Secure Key Seed
[1] random: fast provider: "Intel Secure Key Seed"
[1] random: unblocking device.
[1] ioapic0 <Version 2.0> irqs 0-119
[1] Launching APs: 1 2 3
[1] random: entropy device external interface
[1] wlan: mac acl policy registered
[1] kbd0 at kbdmux0
[1] WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 16.0.
[1] efirtc0: <EFI Realtime Clock>
[1] efirtc0: registered as a time-of-day clock, resolution 1.000000s
[1] smbios0: <System Management BIOS> at iomem 0x75cc3000-0x75cc3017
[1] smbios0: Entry point: v3 (64-bit), Version: 3.5
[1] aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256>
[1] acpi0: <ALASKA A M I >
[1] acpi0: Power Button (fixed)
[1] hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
[1] Timecounter "HPET" frequency 19200000 Hz quality 950
[1] Event timer "HPET" frequency 19200000 Hz quality 550
[1] Event timer "HPET1" frequency 19200000 Hz quality 440
[1] Event timer "HPET2" frequency 19200000 Hz quality 440
[1] Event timer "HPET3" frequency 19200000 Hz quality 440
[1] Event timer "HPET4" frequency 19200000 Hz quality 440
[1] atrtc1: <AT realtime clock> on acpi0
[1] atrtc1: Warning: Couldn't map I/O.
[1] atrtc1: registered as a time-of-day clock, resolution 1.000000s
[1] Event timer "RTC" frequency 32768 Hz quality 0
[1] attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
[1] Timecounter "i8254" frequency 1193182 Hz quality 0
[1] Event timer "i8254" frequency 1193182 Hz quality 100
[1] Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
[1] acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
[1] pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
[1] pci0: <ACPI PCI bus> on pcib0
[1] vgapci0: <VGA-compatible display> port 0x3000-0x303f mem 0x6000000000-0x6000ffffff,0x4000000000-0x400fffffff at device 2.0 on pci0
[1] vgapci0: Boot video device
[1] xhci0: <XHCI (generic) USB 3.0 controller> mem 0x6001110000-0x600111ffff at device 13.0 on pci0
[1] xhci0: 32 bytes context size, 64-bit DMA
[1] xhci0: xECP capabilities <PROTO,PROTO,VEND(c0),LEGACY,VEND(c6),VEND(c7),VEND(c2),DEBUG,VEND(c3),VEND(d1),VEND(ce),VEND(c8),VEND(c9),VEND(ca),VEND(cc),VEND(cd),VEND(d2),VEND(cf),VEND(d3)>
[1] usbus0 on xhci0
[1] usbus0: 5.0Gbps Super Speed USB v3.0
[1] xhci1: <XHCI (generic) USB 3.0 controller> mem 0x6001100000-0x600110ffff at device 20.0 on pci0
[1] xhci1: 32 bytes context size, 64-bit DMA
[1] xhci1: xECP capabilities <PROTO,PROTO,VEND(c0),LEGACY,VEND(c6),VEND(c7),VEND(c2),DEBUG,VEND(c3),VEND(c4),VEND(ce),VEND(c8),VEND(c9),VEND(ca),VEND(cb),VEND(cc),VEND(cd)>
[1] usbus1 on xhci1
[1] usbus1: 5.0Gbps Super Speed USB v3.0
[1] pci0: <memory, RAM> at device 20.2 (no driver attached)
[1] pci0: <simple comms> at device 22.0 (no driver attached)
[1] pcib1: <ACPI PCI-PCI bridge> at device 28.0 on pci0
[1] pci1: <ACPI PCI bus> on pcib1
[1] igc0: <Intel(R) Ethernet Controller I226-V> mem 0x80e00000-0x80efffff,0x80f00000-0x80f03fff at device 0.0 on pci1
[1] igc0: EEPROM V2.13-0 eTrack 0x80000284
[1] igc0: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc0: Using 4 RX queues 4 TX queues
[1] igc0: Using MSI-X interrupts with 5 vectors
[1] igc0: Ethernet address: 00:d0:b4:01:57:0c
[1] igc0: netmap queues/slots: TX 4/1024, RX 4/1024
[1] pcib2: <ACPI PCI-PCI bridge> at device 28.1 on pci0
[1] pci2: <ACPI PCI bus> on pcib2
[1] igc1: <Intel(R) Ethernet Controller I226-V> mem 0x80b00000-0x80bfffff,0x80c00000-0x80c03fff at device 0.0 on pci2
[1] igc1: EEPROM V2.13-0 eTrack 0x80000284
[1] igc1: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc1: Using 4 RX queues 4 TX queues
[1] igc1: Using MSI-X interrupts with 5 vectors
[1] igc1: Ethernet address: 00:d0:b4:01:57:0d
[1] igc1: netmap queues/slots: TX 4/1024, RX 4/1024
[1] pcib3: <ACPI PCI-PCI bridge> at device 28.2 on pci0
[1] pci3: <ACPI PCI bus> on pcib3
[1] igc2: <Intel(R) Ethernet Controller I226-V> mem 0x80800000-0x808fffff,0x80900000-0x80903fff at device 0.0 on pci3
[1] igc2: EEPROM V2.13-0 eTrack 0x80000284
[1] igc2: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc2: Using 4 RX queues 4 TX queues
[1] igc2: Using MSI-X interrupts with 5 vectors
[1] igc2: Ethernet address: 00:d0:b4:01:57:0e
[1] igc2: netmap queues/slots: TX 4/1024, RX 4/1024
[1] pcib4: <ACPI PCI-PCI bridge> at device 28.6 on pci0
[1] pci4: <ACPI PCI bus> on pcib4
[1] igc3: <Intel(R) Ethernet Controller I226-V> mem 0x80500000-0x805fffff,0x80600000-0x80603fff at device 0.0 on pci4
[1] igc3: EEPROM V2.13-0 eTrack 0x80000284
[1] igc3: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc3: Using 4 RX queues 4 TX queues
[1] igc3: Using MSI-X interrupts with 5 vectors
[1] igc3: Ethernet address: 00:d0:b4:01:57:0f
[1] igc3: netmap queues/slots: TX 4/1024, RX 4/1024
[1] pcib5: <ACPI PCI-PCI bridge> at device 29.0 on pci0
[1] pci5: <ACPI PCI bus> on pcib5
[1] nvme0: <Generic NVMe Device> mem 0x81000000-0x81003fff at device 0.0 on pci5
[1] isab0: <PCI-ISA bridge> at device 31.0 on pci0
[1] isa0: <ISA bus> on isab0
[1] hdac0: <Intel Alder Lake-N HDA Controller> mem 0x6001120000-0x6001123fff,0x6001000000-0x60010fffff at device 31.3 on pci0
[1] pci0: <serial bus> at device 31.5 (no driver attached)
[1] acpi_button0: <Sleep Button> on acpi0
[1] cpu0: <ACPI CPU> on acpi0
[1] acpi_button1: <Power Button> on acpi0
[1] acpi_tz0: <Thermal Zone> on acpi0
[1] uart: ns8250: UART FCR is broken (0x1)
[1] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
[1] uart0: console (115200,n,8,1)
[1] uart: ns8250: UART FCR is broken (0x1)
[1] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
[1] acpi_syscontainer0: <System Container> on acpi0
[1] acpi_syscontainer1: <System Container> on acpi0
[1] atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
[1] atrtc0: Warning: Couldn't map I/O.
[1] atrtc0: registered as a time-of-day clock, resolution 1.000000s
[1] atrtc0: Can't map interrupt.
[1] hwpstate_intel0: <Intel Speed Shift> on cpu0
[1] cpufreq0: <CPU frequency control> on cpu0
[1] hwpstate_intel1: <Intel Speed Shift> on cpu1
[1] cpufreq1: <CPU frequency control> on cpu1
[1] hwpstate_intel2: <Intel Speed Shift> on cpu2
[1] cpufreq2: <CPU frequency control> on cpu2
[1] hwpstate_intel3: <Intel Speed Shift> on cpu3
[1] cpufreq3: <CPU frequency control> on cpu3
[1] Timecounter "TSC" frequency 806401148 Hz quality 1000
[1] Timecounters tick every 1.000 msec
[1] ugen1.1: <Intel XHCI root HUB> at usbus1
[1] ugen0.1: <Intel XHCI root HUB> at usbus0
[1] uhub0 on usbus1
[1] uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
[1] ZFS filesystem version: 5
[1] ZFS storage pool version: features support (5000)
[1] uhub1 on usbus0
[1] uhub1: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
[1] nda0 at nvme0 bus 0 scbus0 target 0 lun 1
nda0: <Samsung SSD 970 EVO Plus 500GB 2B2QEXM7 S58SNJ0N613187J>
nda0: Serial Number S58SNJ0N613187J
[1] nda0: nvme version 1.3
nda0: 476940MB (976773168 512 byte sectors)
[1] Trying to mount root from zfs:zroot/ROOT/24.1 []...
[1] uhub1: 3 ports with 3 removable, self powered
[2] uhub0: 16 ports with 16 removable, self powered
[2] pid 30 (zpool) is attempting to use unsafe AIO requests - not logging anymore


[OPNenthu]

Nothing applied could mean that you already have the latest ucode, such as from a BIOS update.

[LOTRouter]

Nothing applied could mean that you already have the latest ucode, such as from a BIOS update.

Makes sense.  I just tried it on an N5105 I have running as well, and I do see the microcode update:

Code Select Expand

[1] CPU microcode: updated from 0x1d to 0x24000026

[Patrick M. Hausen]

Message from x86info-1.31.s03_1:
===>   NOTICE:

This port is deprecated; you may wish to reconsider installing it:

Abandoned upstream, fails to identify anything remotely new according to upstream issue reports.

It is scheduled to be removed on or after 2025-06-30.

This is irrelevant, the "x86info" package will simply go away, eventually. Nothing to worry.

A bit early to ask, but what will be the step to have this converted to the final production version of the package?

It will be included in 26.7.1 - or a hotfix if the team deems it necessary.

[1] CPU microcode: updated from 0x10 to 0x21

So worked as intended.

I followed the pkg steps and rebooted the system. No microcode messages in /var/run/dmesg.boot after boot after seconds or minutes.

After a reboot, right?

I followed the steps and also do not see the microcode load.

To verify you can always do this with the system running:

Code Select Expand

/usr/local/etc/rc.syshook.d/early/40-cpu-microcode
dmesg

You should see a repetition of the CPU info output you normally see at the top of /var/run/dmesg.boot like so:

Code Select Expand

[447] CPU: AMD Ryzen Embedded V1500B                       (2195.88-MHz K8-class CPU)
[447]   Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
[447]   Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
[447]   Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
[...]

If that is the case and you do not see that "updated from ... to ..." there was nothing to update.

Plus of course make sure you have the correct (Intel vs. AMD) version.

HTH,
Patrick

[zz00mm]

When I run the manual syshook on Intel.

root@fw254:~ # /usr/local/etc/rc.syshook.d/early/40-cpu-microcode
/usr/local/etc/rc.syshook.d/early/40-cpu-microcode: /usr/local/etc/rc.d/microcode_update: not found

I did a search for microcode files and don't find the microcode_update anywhere.

I looked on an existing 26.1 and this file doesn't exist.

[Patrick M. Hausen]

root@fw254:~ # /usr/local/etc/rc.syshook.d/early/40-cpu-microcode
/usr/local/etc/rc.syshook.d/early/40-cpu-microcode: /usr/local/etc/rc.d/microcode_update: not found

Try this and repeat, please:

Code Select Expand

pkg install cpu-microcode-rc

I'll update the first posting accordingly. Thank you for reporting.

[zz00mm]

Patrick,
     Thank you for the quick response. Now working.

Zz00mm

[franco]

Try this and repeat, please:

Code Select Expand

pkg install cpu-microcode-rc


Whoops, good call. rc package used to be a direct dependency but it no longer is.  Need to add that to the plugin dependencies.