OPNsense Forum

English Forums => Tutorials and FAQs => Topic started by: meyergru on September 25, 2023, 12:28:17 PM

Title: How to enable automatic microcode updates
Post by: meyergru on September 25, 2023, 12:28:17 PM
As many firewall appliances suffer from neglect by they manufacturers, they often lack current microcode updates.

For example, Intel Alder Lake based CPUs (like N100, N200, N300 and I1215U have a bug that cause instabilities.

As of OpnSense 24.7.2, microcode updates can be enabled for AMD and Intel CPUs by installing the corresponding plugin os-cpu-microcode-amd or os-cpu-microcode-intel (not both!). The update will be done automatically upon next reboot.

The old post is left here for reference:




Since 23.7.9, there should be microcode updates available, which, when enabled, should cure those problems.

Though OpnSense does not have those enabled per default (e.g. it obviously is not applicable for VM installations), the neccessary tools are provided. Here is how:

1. Install the package cpu-microcode - this is a newer variant of the old devcpu-data package. From a shell, call:

echo y | pkg install cpu-microcode


Alongside with this package, the RC scripts and firmwares for both Intel and AMD CPUs will be installed. If you had the devcpu-data package installed before, it will get replaced.

2. Use the web UI to create these two tuneables in /boot/loader.conf:

cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/intel-ucode.bin"


Please double-check that the entires exist. Note that this is only for Intel CPUs, because this addresses only the early boot stage, which is not available for AMD CPUs. This can be fixed in the next step.

3. Add an entry to /etc/rc.conf:

echo 'microcode_update_enable="YES"' >> /etc/rc.conf


This will enable a RC script which handles both Intel and AMD CPUs during boot.

4. (Optional and only available from 23.7.6 on)

pkg install x86info
rehash
x86info -a | fgrep -i microcode


The last command will show you the current microcode version.

5. Reboot or call the CPU firmware update script once with:

service microcode_update start


You will see an error if you skipped step 3.

6. (Optional and only available from 23.7.6 on) Call:

kldload -q cpuctl; x86info -a | fgrep -i microcode

again and you will probably see a different version than in step 4.

Note that these settings and installation of packages will not be part of a configuration backup, so after a hardware migration, you have to repeat most of it.

These instructions are provided as-is, so no warranties whatsoever.
Title: Re: How to enable automatic microcode updates
Post by: franco on September 25, 2023, 07:59:55 PM
It might be possible to wrap this into a trivial plugin, but the recent packaging change in FreeBSD made me happy that we didn't sink time here before. For now I just switched to the new packages. The old ones are no longer available.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: newsense on September 26, 2023, 02:26:43 AM
Thanks meyergru, was just about to look into it (couldn't remember if I installed the meta anywhere) - when I saw Franco's answer which sent me searching for answers.

So...speaking of packages...


root@OPNsense:~ # pkg search cpu-microcode

cpu-microcode-1.0              Meta-package for CPU microcode updates
cpu-microcode-amd-20230808     AMD CPU microcode updates
cpu-microcode-intel-20230808   Intel CPU microcode updates
cpu-microcode-rc-1.0           RC script for CPU microcode updates



It loos like the RC package would be pulled along with the rest


root@OPNsense:~ # pkg install cpu-microcode
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating mimugmail repository catalogue...
mimugmail repository is up to date.
All repositories are up to date.
The following 4 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        cpu-microcode: 1.0 [OPNsense]
        cpu-microcode-amd: 20230808 [OPNsense]
        cpu-microcode-intel: 20230808 [OPNsense]
        cpu-microcode-rc: 1.0 [OPNsense]

Number of packages to be installed: 4



So we'd get this script installed in /usr/local/etc/rc.d/microcode_upddate

Quote#!/bin/sh

# PROVIDE:   microcode_update
# REQUIRE:   root mountcritlocal
# KEYWORD:   nojail
# BEFORE:   SERVERS

#
# Add the following line to /etc/rc.conf to enable flow-capture:
# microcode_update_enable (bool):   Set it to "YES" to update microcode on startup
#               Set to "NO" by default.
# microcode_update_datadir (str):   Directory, microcode updates stored in.
#               Default is "/usr/local/share/cpucontrol"
# microcode_update_cpus (str):      A list of cpus to update on startup, or "ALL" for all.
#               Example: microcode_update_cpus="0 1"
#               Set to "ALL" by default.
# microcode_update_flags (str):      Flags for cpucontrol(8).

. /etc/rc.subr

name="microcode_update"
rcvar=microcode_update_enable
stop_cmd=":"
start_cmd="microcode_update_start"
required_modules="cpuctl"

CMT="/usr/sbin/cpucontrol"

microcode_update_start()
{
   echo "Updating CPU Microcode..."
   if [ "${microcode_update_cpus}" = "ALL" ]; then
      ncpu=`/sbin/sysctl -n hw.ncpu`
      cpus=`jot ${ncpu} 0`;
   else
      cpus=${microcode_update_cpus}
   fi
   for i in ${cpus}; do
      ${CMT} -u ${microcode_update_flags} \
                    -d "${microcode_update_datadir}" /dev/cpuctl${i} 2>&1 | \
                    logger -p daemon.notice -t microcode_update || \
          (echo "Microcode Update Failed." && exit 1)
   done
   if [ "${microcode_update_cpus}" = "ALL" ]; then
                CPUCONTROL_UPDATED=$(cpucontrol -h 2>&1 | grep -q -- -e; echo $?)
                if [ ${CPUCONTROL_UPDATED} -ne 0 ]; then
                        echo "Please update your system in order to update CPU microcode."
                else
         ${CMT} -e /dev/cpuctl0 >/dev/null 2>&1
         if [ $? -ne 0 ]; then
            echo "Re-evalulation of CPU flags Failed."
            exit 1
         fi
                fi
   fi
   echo "Done."
}

load_rc_config $name

# Set default values
if [ -n "${microcode_cpus}" ]; then
   if [ -n "${microcode_update_cpus}" ]; then
      echo "Warning: Ignoring deprecated rc variable, microcode_cpus."
   else
      echo "Warning: rc variable microcode_cpus is deprecated.
Warning: Set microcode_udpate_cpus instead."
      microcode_update_cpus="${microcode_cpus}"
        fi
fi

: ${microcode_update_enable="NO"}
: ${microcode_update_datadir="/usr/local/share/cpucontrol"}
: ${microcode_update_cpus="ALL"}
: ${microcode_update_flags=""}

run_rc_command "$1"


If we're to stick as close as possible to FreeBSD maybe the best path forward would be to include x86info in OPNsense if there aren't a ton of unneeded dependencies or it ? Any future plugin would likely depend on it.


Thinking about this issue with a security hat --- plugin or no plugin, I would much rather have cpu-microcode in the default packages to be installed when deploying OPNsense on AMD64 - and have it pulled automatically in an upcoming 23.7.x

Title: Re: How to enable automatic microcode updates
Post by: franco on September 26, 2023, 07:31:22 AM
I'm not seeing this as a core requirement. The fact that it isn't a plugin yet gives an insight into how important it is for the average user.

And happy to include x64info package given it's not on a heavy dependency chain.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: meyergru on September 26, 2023, 10:33:15 AM
Yes, all needed packages are drawn in alongside cpu-microcode.

I am all for including x86info, as this eliminates the need to fetch a FreeBSD package - although it is only needed for verification (if an update was even done).

As for including cpu-microcode per default: It is not applicable for VM installations, so I second that it is not a core requirement.
Title: Re: How to enable automatic microcode updates
Post by: franco on September 26, 2023, 10:46:47 AM
https://github.com/opnsense/tools/commit/eebe1f8fa
Title: Re: How to enable automatic microcode updates
Post by: newsense on September 26, 2023, 05:37:11 PM
Thanks Franco, I just found 2.7.5 so I guess it was a timing issue and it will become available in 23.7.6


Quote from: meyergruAs for including cpu-microcode per default: It is not applicable for VM installations, so I second that it is not a core requirement.

Whether users at large understand or care about what cpu-microcode does is less important. Including the package in the deployment list would provide a secure default, otherwise is just an option.

Since none of the os-xen, os-vmware or os-virtualbox are defaults cpu-microcode can be marked for removal as soon as a user tries to install any of the aforementioned packages.

The only place where cpu-microcode would need to be excluded would likely be in the BE vm


I rest my case  :)
Title: Re: How to enable automatic microcode updates
Post by: moonman on October 10, 2023, 08:40:00 AM
So this update pkg to 1.20.x and it now gives an error checking for new packages/update. Does anyone know how to rollback to the previous version of pkg. I have the package itself locally from a different system, just can't figure out how to properly downgrade.

Quote from: meyergru on September 25, 2023, 12:28:17 PM
4. (Optional) If you want to verify that the updates are working, you can install the package x86info. This is not contained in OpnSense, therefore you have to edit /usr/local/etc/pkg/repos/FreeBSD.conf to enable FreeBSD repositories temporarily. You can use these commands:

echo "FreeBSD: { enabled: yes }" > /usr/local/etc/pkg/repos/FreeBSD.conf
echo y | pkg install x86info"
echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
rehash
kldload -q cpuctl
x86info -a | fgrep -i microcode


This is the log
root@OPNsense:~ # echo y | pkg install x86info
Updating FreeBSD repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%    7 MiB   6.9MB/s    00:01
Processing entries: 100%
FreeBSD repository update completed. 34062 packages processed.
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
New version of pkg detected; it needs to be installed first.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        pkg: 1.19.2 -> 1.20.7 [FreeBSD]

Number of packages to be upgraded: 1

The process will require 21 MiB more space.
9 MiB to be downloaded.

[1/1] Fetching pkg-1.20.7.pkg: 100%    9 MiB   9.0MB/s    00:01    %
Checking integrity... done (0 conflicting)
[1/1] Upgrading pkg from 1.19.2 to 1.20.7...
[1/1] Extracting pkg-1.20.7: 100%
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating OPNsense repository catalogue...
pkg: No SRV record found for the repo 'OPNsense'
pkg: packagesite URL error for pkg+http://mirror.sfo12.us.leaseweb.net/opnsense/FreeBSD:13:amd64/23.7/latest/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://mirror.sfo12.us.leaseweb.net/opnsense/FreeBSD:13:amd64/23.7/latest/packagesite.txz -- pkg+:// implies SRV mirror type
Unable to update repository OPNsense
Error updating repositories!
root@OPNsense:~ # echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
root@OPNsense:~ # rehash
root@OPNsense:~ # kldload -q cpuctl
root@OPNsense:~ # pkg update
Updating OPNsense repository catalogue...
pkg: No SRV record found for the repo 'OPNsense'
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
pkg: packagesite URL error for pkg+http://mirror.sfo12.us.leaseweb.net/opnsense/FreeBSD:13:amd64/23.7/latest/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://mirror.sfo12.us.leaseweb.net/opnsense/FreeBSD:13:amd64/23.7/latest/packagesite.txz -- pkg+:// implies SRV mirror type
Unable to update repository OPNsense
Error updating repositories!


@meyergru I would delete that part before someone else stumbles upon it.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on October 10, 2023, 12:08:03 PM
@moonman: Whatever is your problem has nothing to do with the installation of x86info package itself, but obviously your pkg has been updated. You can check with: "pkv -v". I have 1.19.2, I do not know why yours was updated at all (mine did not and still does not).

The error message clearly is for the OPNsense repository. That means your configuration is incompatible with the new pkg version.

BTW: I just saw this: https://forum.opnsense.org/index.php?topic=36333.0


First, you should change the repository parameters under /usr/local/etc/pkg/repos/OPNsense.conf in order to be able to use the OPNsense repository again. See: https://www.reddit.com/r/freebsd/comments/15bghf5/pkg_no_srv_record_found_for_the_repo_freebsd/

That is, either remove the "pkg+" part of the repo URL if it is present or change the mirror_type from "SRV" to "NONE". I guess this will be fixed centrally already from what Franco wrote.

Then, you should be able to use "pkg update" again and probably downgrade somehow. Usually, the old package should still be in /var/cache/pkg.

Remember to change back the repo URL or mirror type afterwards or wait for the SRV records being changed as Franco announced.


I still changed the installation step as x86info will be present in 23.7.6 anyhow and get rid of the FreeBSD dependency.
Title: Re: How to enable automatic microcode updates
Post by: franco on October 10, 2023, 12:17:28 PM
Yeah, pkg 1.20 broke a working situation by enforcing a stricter setup. Quarterly updates hit beginning of October. Same as usual. Expect a lot more of the same as people run into it.

We will be staying on 1.19 for a while longer for this and other reasons.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: moonman on October 11, 2023, 05:13:17 AM
Thanks everyone. Running
pkg add -f /var/cache/pkg/pkg-1.19.2.pkg
fixed it for me
Title: Re: How to enable automatic microcode updates
Post by: Ground_0 on October 30, 2023, 01:12:35 PM
Not quite a necro, I hope..

1. How vulnerable to microcode attacks are those of us using a bare metal install of OPNsense with no remote login?
2. Should those of us on bare metal with no remote login consider enabling automatic microcode updates?

My (limited) understanding is that side channel attacks like Spectre and Meltdown are a threat if you run applications on a system designed for multiple untrusted users, like in a webserver environment. I also fear that configuring automatic updates (IF they are not strictly necessary on an OPNsense machine without remote login) might lead to "entropy" wherein one might be met with unexpected breakage with updates and forget how to address it. It seems that the packages and configuration for this have changed over time and may continue to change and require manual intervention.

Any advice appreciated.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on October 30, 2023, 05:00:07 PM
If you limit the scope to these specific CPU bugs, you could argue that nobody could sniff into running processes anyway, given that they can start no program on the firewall. However, some microcode updates address stability, performance and / or real CPU bugs. Here (https://forum.opnsense.org/index.php?topic=36919.0;topicseen) is an example.

They were invented after the infamous Intel Pentium FDIV (https://en.wikipedia.org/wiki/Pentium_FDIV_bug) and F00F bugs (https://en.wikipedia.org/wiki/Pentium_F00F_bug).

Some of these updates are not even discussed openly (https://www.theregister.com/2023/05/15/intel_mystery_microcode/).

AMD is known to often address issues with newer AGESA versions (https://www.msi.com/blog/do-agesa-1.0.0.7c-bios-and-msi-high-efficiency-mode-improve-gaming-performance). Intel is as well (https://www.extremetech.com/computing/intel-found-at-fault-for-recent-bsods-on-raptor-lake-cpus). Alas, both companies sometimes miss to publish new fixes in time, thus official Linux and FreeBSD microcode repositories lag behind. There is a repository on Github (https://github.com/platomav/CPUMicrocodes) that contains all known microcodes, but they are not in a form to use them directly.

For these reasons, I apply such upgrades even if there are no specific security issues, just for good measure. Sometimes I even patch the firmwares to include fixes for known defects myself, but that is beyond the scope of what normally needs to be done.
Title: Re: How to enable automatic microcode updates
Post by: Seimus on June 01, 2024, 12:46:07 PM
Thank you for the guide,

I did this for N5105 as well N100 worked flawlessly.

Regards,
S.
Title: Re: How to enable automatic microcode updates
Post by: hushcoden on June 04, 2024, 12:15:52 PM
I've just installed that according to the guide, and if I understood correctly, it can be done in three steps:

1. From a shell: echo y | pkg install cpu-microcode

2. Use the web UI to create these two tuneables in /boot/loader.conf:

     cpu_microcode_load="YES"

     cpu_microcode_name="/boot/firmware/intel-ucode.bin"   -> for Intel CPUs

     or

     cpu_microcode_name="/boot/firmware/amd-ucode.bin"   -> for AMD CPUs

3. Reboot

Is that right?

Tia.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on June 04, 2024, 02:48:01 PM
Quote from: hushcoden on June 04, 2024, 12:15:52 PM
I've just installed that according to the guide, and if I understood correctly, it can be done in three steps:

1. From a shell: echo y | pkg install cpu-microcode

2. Use the web UI to create these two tuneables in /boot/loader.conf:

     cpu_microcode_load="YES"

     cpu_microcode_name="/boot/firmware/intel-ucode.bin"   -> for Intel CPUs

     or

     cpu_microcode_name="/boot/firmware/amd-ucode.bin"   -> for AMD CPUs

3. Reboot

Is that right?

Tia.
No.

I explained this in detail. You cannot load AMD microcode in the early stage.

Also, you mixed up /etc/rc.loader.conf and /boot/loader.conf, as well as the syntax of what must be in there.

Just follow the instructions.
Title: Re: How to enable automatic microcode updates
Post by: grimelog on June 07, 2024, 10:48:53 AM
Wouldn't having this in a plugin improve the security of OPNsense?

I don't like having to venture into the repos of non-hardened BSD to install microcode updates. But, definitely running on bare metal those updates could have security and performance benefits. Is there any means of including this in an official plugin since we do not need this for VMs.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on June 07, 2024, 12:54:07 PM
There is no need for the FreeBSD repos as the Package is in OpnSense already.

Other than that, open a feature request on GitHub.
Title: Re: How to enable automatic microcode updates
Post by: alirx on July 15, 2024, 11:33:27 AM
I'm totally new to opnsense and having trouble with step 2.

a) (https://i.imgur.com/PH89pdf.png)
b) (https://i.imgur.com/WqyhIgv.png)

Which is the right way to do it? I don't understand how to point the values "cpu_microcode_load="YES"" and "cpu_microcode_name="/boot/firmware/intel-ucode.bin"" to "/boot/loader.conf" specifically. Just dont want to mess things up.

I'd appreciate some guidance.
Title: Re: How to enable automatic microcode updates
Post by: Patrick M. Hausen on July 15, 2024, 12:45:37 PM
Tunable: cpu_microcode_load
Value: YES
Description: whatever

All without quotes, = signs or similar.
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 04, 2024, 11:04:32 AM
It looks like in the future AMD variant can also be "early loaded" according to recent cpu-microcode message on OpnSense 24.7_9:

Quote1. Early loading.
   This method does not use the RC script included here.
   This is the preferred method, because it ensures that any CPU features
   added or removed by a microcode update are visible to the kernel by
   applying the update before the kernel performs CPU feature detection.

   To enable updates using early loading, add the following lines to
   /boot/loader.conf:

   cpu_microcode_load="YES"

   and the appropriate one of these lines:

   cpu_microcode_name="/boot/firmware/intel-ucode.bin"
   cpu_microcode_name="/boot/firmware/amd-ucode.bin"

   The microcode update will be loaded when the system is rebooted.

   AMD systems running FreeBSD prior to 2024-02-22 snapshot
   34467bd76 only support late loading.

Now I don't know what exact Frisbee version OpnSense currently happens to have, but definitely looks promising overall for the future <3
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 04, 2024, 11:22:14 AM
Interesting find, but it does not apply to the current OpnSense version yet, since it is based on FreeBSD 14.1, released on 2023-06-04, so it does not include the neccessary changes (unless Franco includes the patches).
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 04, 2024, 12:16:21 PM
I do understand that, hencewhy such future prospecting wording :)
Title: Re: How to enable automatic microcode updates
Post by: franco on August 05, 2024, 09:00:49 AM
The issue with https://cgit.freebsd.org/src/commit/?id=34467bd76 as I see it is that there is no intention of bringing it to older FreeBSD releases than what is to be 15.0 some day (no MFC annotation and not on stable/14).

It also doesn't cleanly apply to FreeBSD 14.1 as a single commit. This means there is no exposure on FreeBSD 14 and therefore also no intentional bugfixing if there are problems with it.

So I'm not saying this won't ever be in OPNsense based on FreeBSD 14, but I am saying it's not because we don't want it. ;)


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 05, 2024, 06:05:32 PM
Translation from franco's text:

This will be natural part of OPNsense when FreeBSD 15 major update hits OPNsense in the future.  8)
Title: Re: How to enable automatic microcode updates
Post by: franco on August 05, 2024, 08:26:42 PM
Yes, but also have fun trying it out:

# opnsense-update -zkr 24.7-amd

It's on a separate branch and will stay there for a bit in any case.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 06, 2024, 10:48:03 AM
Hmm, what's in there? or is there an place to see? =) I mean I'd love to take na peek before I commit to something, but in general I could ;D

Quote from: franco on August 05, 2024, 08:26:42 PM
Yes, but also have fun trying it out:

# opnsense-update -zkr 24.7-amd

It's on a separate branch and will stay there for a bit in any case.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 06, 2024, 11:20:23 AM
That kernel contains among other improvements the patches to enable early microcode loading for AMD processors. I cannot test it because my boxes all have Intel CPUs.
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 06, 2024, 12:15:28 PM
Quote from: franco on August 05, 2024, 08:26:42 PM
Yes, but also have fun trying it out:

# opnsense-update -zkr 24.7-amd

It's on a separate branch and will stay there for a bit in any case.


Cheers,
Franco

Allright, I am testing this with DEC2752, 24.7_9 as "base",

olmari@router:~ $ uname -a
FreeBSD router.huutoniemi 14.1-RELEASE-p2 FreeBSD 14.1-RELEASE-p2 amd_early-n267771-478b7ed2f02d SMP amd64


I put early loading into tunables, which produced these in /boot/loader.conf:

# dynamically generated tunables settings follow
cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/amd-ucode.bin"


/etc/rc.conf.d/cpu_microcode has the

microcode_update_enable="YES"

At least microcode still loads, No idea is it early or late stage =)

root@router:~ # kldload -q cpuctl; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b
Title: Re: How to enable automatic microcode updates
Post by: franco on August 06, 2024, 12:20:31 PM
The branch is https://github.com/opnsense/src/tree/amd_early

Might be worth defanging the late loading code to verify the early load works now...


Cheeers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 06, 2024, 12:39:40 PM
Quote from: franco on August 06, 2024, 12:20:31 PM
The branch is https://github.com/opnsense/src/tree/amd_early

Might be worth defanging the late loading code to verify the early load works now...


Cheeers,
Franco

After deleting /etc/rc.conf.d/cpu_microcode and reboot, microcode seems to still load:

root@router:~ # kldload -q cpuctl; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b

Title: Re: How to enable automatic microcode updates
Post by: olmari on August 06, 2024, 01:03:26 PM
Looking from dmesg, I get practically this, I don't know is it good or bad:

root@router:~ # dmesg | grep microcode -A 20 -B 1
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2196.04-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
Title: Re: How to enable automatic microcode updates
Post by: franco on August 06, 2024, 01:17:28 PM
> CPU microcode: no matching update found

This is a bit weird. Do microcode updates persist through warm reboots?


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 06, 2024, 01:21:26 PM
If they do, then testing takes "halt" for couple of hours until I can go power cycle the router physically, then we'll see for sure.
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 07, 2024, 06:35:21 PM
After hardware power cycle, the /etc/rc.conf.d/cpu_microcode file deleted (to make sure no late loading happens), microcode still seems to be applied and I still can't find anything proper on dmesg...

So I'd say early load works, but not everything is 100% maybe, possibly :D

root@router:~ # dmesg | grep microcode -A 20 -B 1
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2195.94-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222998528 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.

root@router:~ # kldload -q cpuctl; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 07, 2024, 06:40:32 PM
I would guess that this message:

CPU microcode: no matching update found

means that there is nothing to update. This could be because of:

a. The BIOS already has the latest update included.
b. The microcode has persisted over a warm reset.
c. Something is not working, like if you failed to modify step 2 from my instructions to:


cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/[b]amd[/b]-ucode.bin"


What happens if you try to apply the update afterwards via the late loading method? Does it update?
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 07, 2024, 06:51:25 PM
a) In theory yes, in practice I'd say no because before any microcode update attempt of any kind I don't see the microcode patch level message at all, I suppose I could try to confirm this 100%

b) That is why I now tested cold power cycle in the last message (with driving to on premises :P ). same result.

c) correct lines in tunables -> loader, as per my earlier post..
Title: Re: How to enable automatic microcode updates
Post by: franco on August 07, 2024, 06:56:19 PM
Does it take a new microcode update package to ship then maybe?


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 07, 2024, 08:06:13 PM
Hmm, even if I change cpu_microcode_load="NO" I get same results... So the "a" might be true after all..

root@router:~ # dmesg | grep microcode -A 20 -B 1
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2195.94-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222998528 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
--
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2196.06-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222994432 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.

root@router:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b


EDIT: Though again "only" warm boot, as devide is still in remote location, so.. meh.. :D
Title: Re: How to enable automatic microcode updates
Post by: franco on August 07, 2024, 08:14:26 PM
Ok, judging by the code:

https://github.com/opnsense/src/blob/478b7ed2f02/sys/x86/x86/ucode.c#L90-L111

It means no update was carried out for one reason or another. The actual message we want to see is "CPU microcode: updated from x to y" but seeing that message for AMD might mean the code is there and working.

The sanity test is going to the stock kernel if that stuff logs at all there? If not then the code is there and executing.

So we may indeed need to wait for a new microcode package update.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 07, 2024, 08:24:40 PM
I tried with stock kernel, it indeed seems that microcode has been stayed the same, again hard problem is that it is "only" warm reboot...

olmari@router:~ $ uname -a
FreeBSD router.huutoniemi 14.1-RELEASE-p2 FreeBSD 14.1-RELEASE-p2 stable/24.7-n267758-4ad7ad40bc77 SMP amd64

root@router:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b

root@router:~ # dmesg | grep microcode -A 20 -B 1
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2195.94-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222998528 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
--
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2196.06-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222994432 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
Title: Re: How to enable automatic microcode updates
Post by: franco on August 07, 2024, 08:31:27 PM
That's a bit strange as it also prints the message. Hard to debug based off of that. ;)


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 07, 2024, 08:34:14 PM
Sorry! It did _not_ print anything about microcode in latest reboot. I grepped microcode and it showed the 2 earlier boots... last one does NOT have microcode in it, ofcourse it didn't show up with grepping...

VT(vga): resolution 640x480
CPU: AMD Ryzen Embedded V1500B                       (2196.03-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)


ADD: Microcode patch level: 0x810100b still, so there is that..
Title: Re: How to enable automatic microcode updates
Post by: franco on August 07, 2024, 08:39:35 PM
Ah, ok, we should wait for a new microcode update package then and call it a day :)
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 07, 2024, 10:17:48 PM
I got my hands on the DEC750 I have and resetted it to stock, cold booted, and indeed with at the least latest bios for the moment (version 30), it seems to already have the latest microcode:

root@OPNsense:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
CPU0: local APIC error 0x80
Microcode patch level: 0x810100b
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 08, 2024, 12:52:38 AM
And indeed in BIOS itself there is this line:

Ucode Patch Version                        810100B

So that answers that :D
Title: Re: How to enable automatic microcode updates
Post by: franco on August 12, 2024, 02:54:08 PM
@olmari: here's the latest microcode package to test with

# opnsense-revert -z cpu-microcode-amd

# pkg query %v cpu-microcode-amd
20240810


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 12, 2024, 04:03:21 PM
Fixing Sinkclose (CVE-2023-31315), I presume?
Title: Re: How to enable automatic microcode updates
Post by: franco on August 12, 2024, 04:33:11 PM
Probably. Haven't double-checked.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: 2HgRyz13 on August 13, 2024, 10:03:33 AM
I, too, would like to know the estimated fix date for the SinkClose vulnerability in the EPYC 3201 in my OpnSense DEC850.

My DEC850 is new. I configured it but haven't deployed it yet. I'll wait until SinkClose is patched.

AMD's advisory seems to list the Embedded EPYC 3000 [series] "target" as Oct 2024, if I read it correctly:
https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7014.html

Then I assume OpnSense will need X weeks to incorporate the fix.

So maybe Oct at the earliest but Nov or Dec more likely.

I can wait but was looking forward to installing the DEC850 this week. :(
Title: Re: How to enable automatic microcode updates
Post by: franco on August 13, 2024, 10:16:56 AM
> I, too, would like to know the estimated fix date

To be frank same as any other vulnerability... likely next minor release.

As for the previous one not fixing it. I can't ship a fix that I didn't have last week.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 13, 2024, 10:20:29 AM
FWIW, I doubt that this new update has addressed CVE-2023-31315 yet. E.g., Debian will do this for Bookworm with a point release, which is due in a few months and has only updated Debian Sid so far. I have tested that update and found that the microcode version did not change for my AMD Ryzen 5700G from the previous version. Yet, the bulletin states that this CPU is affected.

Also, Platomav  (https://github.com/platomav/CPUMicrocodes)has only one update from last week for AMD - there should be more than that if the fixes had been published.

Thus, I am not even sure if it will be a microcode update alone that will fix it. AFAIK, AGESA is a little bit more than just the microcode.

AMD has chosen to name the due update 1.2.0.cb while most OEMs still have 1.2.0.C in their BIOSes. Asus really calls it 1.2.0.ca, but I failed to find any BIOS update with 1.2.0.cb from the big OEMs.

So, I guess we will have to wait.
Title: Re: How to enable automatic microcode updates
Post by: franco on August 13, 2024, 10:23:57 AM
The ports vulnerability database says https://vuxml.freebsd.org/freebsd/7d631146-5769-11ef-b618-1c697a616631.html


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 13, 2024, 10:31:10 AM
Interesting. The Debian version for Sid is 3.20240710.1, so obviously a month too early. So FreeBSD seems ahead this time... ;-)
Title: Re: How to enable automatic microcode updates
Post by: franco on August 13, 2024, 02:12:09 PM
While we are at it... https://github.com/opnsense/plugins/commit/77ecf1eb8


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 14, 2024, 05:18:08 AM
Maybe I'm dense, but on 24.7.1 (stock and amd-early kernel) I get no updated cpu-microcode-amd package..

root@router:~ # pkg update
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
root@router:~ # pkg search cpu-microcode
cpu-microcode-1.0_1            Meta-package for CPU microcode updates
cpu-microcode-amd-20240116     AMD CPU microcode updates
cpu-microcode-intel-20240531   Intel CPU microcode updates
cpu-microcode-rc-1.0_2         RC script for CPU microcode updates
root@router:~ # pkg query %v cpu-microcode-amd
root@router:~ #
Title: Re: How to enable automatic microcode updates
Post by: newsense on August 14, 2024, 06:01:50 AM
It will be available in 24.7.2, likely next week.
Title: Re: How to enable automatic microcode updates
Post by: franco on August 14, 2024, 06:42:30 AM
That's why I said use opnsense-revert to grab the snapshot:

# opnsense-revert -z cpu-microcode-amd

# pkg query %v cpu-microcode-amd
20240810

:)


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 14, 2024, 07:42:56 PM
root@router:~ # opnsense-revert -z cpu-microcode-amd
Package 'cpu-microcode-amd' is not installed
root@router:~ # pkg query %v cpu-microcode-amd
root@router:~ # pkg install cpu-microcode-amd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
cpu-microcode-amd: 20240116
cpu-microcode-rc: 1.0_2

Number of packages to be installed: 2

60 KiB to be downloaded.

Proceed with this action? [y/N]:
root@router:~ # pkg install cpu-microcode-amd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
cpu-microcode-amd: 20240116
cpu-microcode-rc: 1.0_2

Number of packages to be installed: 2

60 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/2] Fetching cpu-microcode-rc-1.0_2.pkg: 100%    3 KiB   2.6kB/s    00:01   
[2/2] Fetching cpu-microcode-amd-20240116.pkg: 100%   58 KiB  59.4kB/s    00:01   
Checking integrity... done (0 conflicting)
[1/2] Installing cpu-microcode-rc-1.0_2...
[1/2] Extracting cpu-microcode-rc-1.0_2: 100%
[2/2] Installing cpu-microcode-amd-20240116...
[2/2] Extracting cpu-microcode-amd-20240116: 100%
=====
Message from cpu-microcode-rc-1.0_2:

--
This port includes an RC script, which is one of two methods to update
the CPU microcode on a FreeBSD system.

1. Early loading.
   This method does not use the RC script included here.
   This is the preferred method, because it ensures that any CPU features
   added or removed by a microcode update are visible to the kernel by
   applying the update before the kernel performs CPU feature detection.

   To enable updates using early loading, add the following lines to
   /boot/loader.conf:

   cpu_microcode_load="YES"

   and the appropriate one of these lines:

   cpu_microcode_name="/boot/firmware/intel-ucode.bin"
   cpu_microcode_name="/boot/firmware/amd-ucode.bin"

   The microcode update will be loaded when the system is rebooted.

   AMD systems running FreeBSD prior to 2024-02-22 snapshot
   34467bd76 only support late loading.


2. Late loading.
   This method, which does use the RC script included here, is enabled by
   adding the following line to /etc/rc.conf:

   microcode_update_enable="YES"

   The microcode update is then applied upon reboot or when the microcode
   update service is run via:

   # service microcode_update start

   If the CPU requires a microcode update, a console message such as the
   following will appear:

   Updating CPU Microcode...
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl0 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl2 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl4 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl6 from rev 0x17 to rev 0x22... done.
   Done.

It is safe to enable both methods.
=====
Message from cpu-microcode-amd-20240116:

--
Refer to the cpu-microcode-rc installation notes to enable AMD microcode
updates.
root@router:~ # opnsense-revert -z cpu-microcode-amd
Fetching cpu-microcode-amd.pkg: ... done
Verifying signature with trusted certificate pkg.opnsense.org.20240611... done
cpu-microcode-amd-20240116: already unlocked
Installing cpu-microcode-amd-20240810...
package cpu-microcode-amd is already installed, forced install
Extracting cpu-microcode-amd-20240810: 100%
=====
Message from cpu-microcode-amd-20240810:

--
Refer to the cpu-microcode-rc installation notes to enable AMD microcode
updates.
root@router:~ # pkg query %v cpu-microcode-amd
20240810

Dmesg: CPU microcode: no matching update found

root@router:~ # kldload -q cpuctl; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b


So, either something does not load correctly, or for Ryzen v1500b there is no microcode update in this round. My bet is in the latter by preliminary looks.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 14, 2024, 09:06:44 PM
Indeed. This CVE-2023-31315 info page says (https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7014.html) that Ryzen Embedded V1000 series and the V1500b, also dubbed YE1500C4T4MFB is expected to get an update with a target date of Oct. 2024.
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 14, 2024, 09:19:29 PM
But this also looks like the Amd early loading system itself works, at least the "no updates found" part. Like Fitch Franco (at forum side :D ) said earlier, I think this is now mostly waiting game for updated suitable microcode package for the actual code loading part to happen on this system :)
Title: Re: How to enable automatic microcode updates
Post by: fadern on August 21, 2024, 04:58:51 PM
Hi,
How will the manual setup work/coexist with the new plugin?
Title: Re: How to enable automatic microcode updates
Post by: franco on August 21, 2024, 05:36:47 PM
Yep, doesn't matter as long as you install the right plugin for your CPU.

Eventually we want to release the early AMD loading code in 24.7.x and remove the late loading from the plugins. So for now the plugins use both load points simultaneously. This means people can further test the 24.7-amd kernel early loading.

I might throw out a new kernel for the early AMD loading. But not today.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: Wrigleys on August 21, 2024, 08:26:30 PM
Hi all,

I can confirm that after reverting the tuneables to default and deleting the line "microcode_update_enable="YES"" in /etc/rc.conf, the microcode update will still be performed after reboot.

Many thanks franco and the whole team for another seamless and struggle-free update.

All the best,
Wrigleys
Title: Re: How to enable automatic microcode updates
Post by: franco on August 21, 2024, 08:53:06 PM
Hehe that was the plan, nice :)


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: newsense on August 22, 2024, 01:51:54 AM
It is best to make sure os-cpu-microcode-{intel,amd} is installed going forward on bare metal systems.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 22, 2024, 09:21:13 AM
Yes, but only either one of them, as they are mutually exclusive. If you try to install both, you get strange results (i.e. last one wins and the other package is then "missing").
Title: Re: How to enable automatic microcode updates
Post by: franco on August 22, 2024, 09:45:35 AM
"Missing" in the plugins list? I thought it was just going to uninstall the other one. Then again we didn't have conflicting plugins since os-quagga and os-frr were in the packages alongside for a bit...
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 22, 2024, 10:14:38 AM
In my case, it was uninstalled and then showed up as missing. I had to uninstall the wrong one again and then install the correct one. Maybe the "missing" was an artifact?
Title: Re: How to enable automatic microcode updates
Post by: cookiemonster on August 22, 2024, 10:19:37 AM
Nice addition. Will it let the admin know when attempting on a non-suitable system, like a Virtual Machine?
I know he/she *should* know, but we've seen people with VMs and misunderstandings.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 22, 2024, 10:26:46 AM
Why? It does not hurt. As little as when you choose the wrong version (i.e. AMD vs. Intel). I think many people will stumble over the "install both" hurdle.

In a perfect world (TM), a single plugin would automatically choose if it does nothing (because VM), or installs Intel or AMD versions on installation.
Title: Re: How to enable automatic microcode updates
Post by: franco on August 22, 2024, 11:00:35 AM
The point of contention is:

% git grep microcode_name
sysutils/cpu-microcode-amd/src/etc/rc.loader.d/40-cpu-microcode:cpu_microcode_name="/boot/firmware/amd-ucode.bin"
sysutils/cpu-microcode-intel/src/etc/rc.loader.d/40-cpu-microcode:cpu_microcode_name="/boot/firmware/intel-ucode.bin"

Short of adding more logic to find out what CPU microcode is applicable or adding a GUI toggle this is beyond redemption.

It would probably make for sense for FreeBSD to use it this way... expecially since the kernel code knows if it's executing Intel or AMD microcode because they are separate... ;)

sysutils/cpu-microcode-amd/src/etc/rc.loader.d/40-cpu-microcode:cpu_microcode_amd="/boot/firmware/amd-ucode.bin"
sysutils/cpu-microcode-intel/src/etc/rc.loader.d/40-cpu-microcode:cpu_microcode_intel="/boot/firmware/intel-ucode.bin"

And you have to consider this means pulling irrelevant updates of the other package all the time because it is a hard dependency even if not used.

Funny the late loading code does this automatically without tripping over itself. But I would rather lock this to only use the early loading once the AMD support is in 24.7.x.

We may add the AMD one to our factory images. For now this approach seems the path of least resistance.
Title: Re: How to enable automatic microcode updates
Post by: cookiemonster on August 22, 2024, 11:03:17 AM
Why? - because this is something I want to do when I am around not automatically for a VM especially.
Yes it can hurt. I have had a VM that would crash after a few hours of uptime and took me a while to track the problem. It was fwupd scheduled (not by me) triggering a fwupd update.
Yes it is a different OS but me, no I don't what firmware updates to happen unless I trigger them. Yes they can hurt.
For me is easy in this case, not install the plugin.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 22, 2024, 11:15:16 AM
That is somewhat besides my point: fwupd updates firmwares and obviously can do harm. You cannot update microcode from inside a VM. It just does not do anything - which includes harm.

Also there is no cron or systemd job that regularly applies this, as microcode updates are done once (either at kernel start for early loading or - for now only - during the boot process).
Title: Re: How to enable automatic microcode updates
Post by: cookiemonster on August 22, 2024, 11:21:54 AM
Well I will have to respectfully disagree with you there meyergru. The firmware update was problematic for me from a VM, it crashed it instead of not doing anything.
but as I said, easy for me to not install the plugin. The comment and input here is that the plugin should include a warning by default for users of OPN in a VM.
Title: Re: How to enable automatic microcode updates
Post by: meyergru on August 22, 2024, 11:29:21 AM
But you wrote about problems with fwupd - AFAIK, that is a Linux tool to update device firmwares, not microcodes.

Any decent hypervisor will block CPU microcode updates on VM guests (I know for a fact VMware does that), so you should be safe trying this inside a VM (in vain) - unless my information w/r to this is wrong. Of course, the update code will only try if the emulated CPU matches, anyway, so for KVM, it would have to be "host", not any emulated type (and I bet that the microcode update is not emulated, either).
Title: Re: How to enable automatic microcode updates
Post by: Taunt9930 on August 25, 2024, 12:27:11 PM
How would effectively roll-back these steps / remove the functionality ready for the built-in version upcoming in a release. Do I need to, or can I leave as-is without issue/conflict?
Title: Re: How to enable automatic microcode updates
Post by: franco on August 26, 2024, 08:59:30 AM
Quote from: franco on August 21, 2024, 05:36:47 PM
Yep, doesn't matter [what you did manually to set it up before] as long as you install the right plugin for your CPU.


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: franco on August 27, 2024, 03:22:53 PM
I updated the 24.7-amd kernel to hold the early loading patches on top of what is probably going to be a good 24.7.3 kernel for further testing. I think we can introduce them in the next 24.7.x kernel round. Hopefully that's not going to be 24.7.4. ;)


Cheers,
Franco
Title: Re: How to enable automatic microcode updates
Post by: olmari on August 28, 2024, 09:06:54 PM
Indeed only thing at the moment preventing me to test happy-path of Amd early loading is lack of more fresh microcode, indeed otherwise I'm happy with what we have had so far (the check and "no patch to update" as DEC750 already has the current latest =)
Title: Re: How to enable automatic microcode updates
Post by: 2HgRyz13 on September 10, 2024, 04:02:16 AM
FYI — version 24.7.2 (August 21, 2024), opnsense release doc:

"As a special note we now have native CPU microcode update plugins for either AMD or Intel to install from the GUI. Apart from a reboot these plugins require no further user interaction and will keep the applicable microcode at the latest known version as shipped in the packages repository."
Title: Re: How to enable automatic microcode updates
Post by: 2HgRyz13 on October 01, 2024, 03:17:43 AM
After updating my DEC850 's (AMD EPYC) firmware and installing the new microcode plugin, I used SSH to see the running microcode version:

0x800126f

Unfortunately, I forgot to check the microcode version before I installed the plugin.

Is 0x800126f the latest?

When was it released?

If released recently, did it fixed the SinkClose vulnerability?

I didn't find the answers in numerous online searches.

I'm still waiting for the sinkclose fix before I deploy my new DEC850.


UPDATE: AMD's bulletin (https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7014.html) lists microcode version 0x0800126F 2024-05-03 as "Mitigation Option 2" for fixing CVE-2023-31315 (sinkclose) for 1st Gen AMD EPYC processors (including 3201 in DEC 850, I assume) listed in the Data Center section. So it looks like  0x800126f fixed the flaw a long time ago. Also, 0x800126f is listed here (https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/amd-ucode/README) under microcode_amd_fam17h.bin in the line Family=0x17 Model=0x01 Stepping=0x02: Patch=0x0800126f and appears to be the latest version. I found 0x0800126e (the previous version) mentioned here (https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/amd-ucode/README) posted on 9-19-2022.
Title: Re: How to enable automatic microcode updates
Post by: Patrick M. Hausen on October 01, 2024, 03:28:12 AM
Why do you think sinkclose is relevant for a dedicated firewall appliance? It's not.
Title: Re: How to enable automatic microcode updates
Post by: olmari on October 10, 2024, 01:46:23 AM
I see the 24.7.6 having early loading landed in it, thank you! Only thing now missing from the "final test" for my devices is the actual microcode release by AMD. xD