Messages

I don't think the routers own NVMe would be best target nor source.. Or generally any single NVMe for that matter... If you wanna test the networking capabilities of the router, do it with, say, 2 hosts using SPF+ each and whatever perf test you manage to do that just throws data to /dev/null in practice... Otherwise you're bot testing networking performance quite at all...

I see the 24.7.6 having early loading landed in it, thank you! Only thing now missing from the "final test" for my devices is the actual microcode release by AMD. xD

Indeed only thing at the moment preventing me to test happy-path of Amd early loading is lack of more fresh microcode, indeed otherwise I'm happy with what we have had so far (the check and "no patch to update" as DEC750 already has the current latest =)

Well, I know this is/was an long shot... Many, especially low-end consumer routers, has every led behind GPIO practically, whole another thing is that is it ever exposed with OEM firmware, with Openwrt it is usually available in those =)

Then another thing is that indeed sometimes NIC/PHY drivers/firmware/xxx has undocumented features where led control is one of them...

Ofcourse I know there exists MacGyver solutions with Jesus-tape and pineapples and tree sap, but software control would have been awesome ;P

For f**k sakes...

But this also looks like the Amd early loading system itself works, at least the "no updates found" part. Like Fitch Franco (at forum side :D ) said earlier, I think this is now mostly waiting game for updated suitable microcode package for the actual code loading part to happen on this system :)

I have two DEC750 and one DEC2752 where one of the 750s is especialyl at parade place at domestic enviroment. I'd so wish there would be an way to basically turn all lights/leds off... Optionally there could be some "one light blinks every minute to show it is on" if such is desired, but generally if I could jsut turn of the blinkenlights and sleep better at nights would be awesome. :P

Code Select Expand

root@router:~ # opnsense-revert -z cpu-microcode-amd
Package 'cpu-microcode-amd' is not installed
root@router:~ # pkg query %v cpu-microcode-amd
root@router:~ # pkg install cpu-microcode-amd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
cpu-microcode-amd: 20240116
cpu-microcode-rc: 1.0_2

Number of packages to be installed: 2

60 KiB to be downloaded.

Proceed with this action? [y/N]:
root@router:~ # pkg install cpu-microcode-amd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
cpu-microcode-amd: 20240116
cpu-microcode-rc: 1.0_2

Number of packages to be installed: 2

60 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/2] Fetching cpu-microcode-rc-1.0_2.pkg: 100%    3 KiB   2.6kB/s    00:01   
[2/2] Fetching cpu-microcode-amd-20240116.pkg: 100%   58 KiB  59.4kB/s    00:01   
Checking integrity... done (0 conflicting)
[1/2] Installing cpu-microcode-rc-1.0_2...
[1/2] Extracting cpu-microcode-rc-1.0_2: 100%
[2/2] Installing cpu-microcode-amd-20240116...
[2/2] Extracting cpu-microcode-amd-20240116: 100%
=====
Message from cpu-microcode-rc-1.0_2:

--
This port includes an RC script, which is one of two methods to update
the CPU microcode on a FreeBSD system.

1. Early loading.
   This method does not use the RC script included here.
   This is the preferred method, because it ensures that any CPU features
   added or removed by a microcode update are visible to the kernel by
   applying the update before the kernel performs CPU feature detection.

   To enable updates using early loading, add the following lines to
   /boot/loader.conf:

   cpu_microcode_load="YES"

   and the appropriate one of these lines:

   cpu_microcode_name="/boot/firmware/intel-ucode.bin"
   cpu_microcode_name="/boot/firmware/amd-ucode.bin"

   The microcode update will be loaded when the system is rebooted.

   AMD systems running FreeBSD prior to 2024-02-22 snapshot
   34467bd76 only support late loading.


2. Late loading.
   This method, which does use the RC script included here, is enabled by
   adding the following line to /etc/rc.conf:

   microcode_update_enable="YES"

   The microcode update is then applied upon reboot or when the microcode
   update service is run via:

   # service microcode_update start

   If the CPU requires a microcode update, a console message such as the
   following will appear:

   Updating CPU Microcode...
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl0 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl2 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl4 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl6 from rev 0x17 to rev 0x22... done.
   Done.

It is safe to enable both methods.
=====
Message from cpu-microcode-amd-20240116:

--
Refer to the cpu-microcode-rc installation notes to enable AMD microcode
updates.
root@router:~ # opnsense-revert -z cpu-microcode-amd
Fetching cpu-microcode-amd.pkg: ... done
Verifying signature with trusted certificate pkg.opnsense.org.20240611... done
cpu-microcode-amd-20240116: already unlocked
Installing cpu-microcode-amd-20240810...
package cpu-microcode-amd is already installed, forced install
Extracting cpu-microcode-amd-20240810: 100%
=====
Message from cpu-microcode-amd-20240810:

--
Refer to the cpu-microcode-rc installation notes to enable AMD microcode
updates.
root@router:~ # pkg query %v cpu-microcode-amd
20240810

Dmesg: CPU microcode: no matching update found

root@router:~ # kldload -q cpuctl; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b

So, either something does not load correctly, or for Ryzen v1500b there is no microcode update in this round. My bet is in the latter by preliminary looks.

Maybe I'm dense, but on 24.7.1 (stock and amd-early kernel) I get no updated cpu-microcode-amd package..

Code Select Expand

root@router:~ # pkg update
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
root@router:~ # pkg search cpu-microcode
cpu-microcode-1.0_1            Meta-package for CPU microcode updates
cpu-microcode-amd-20240116     AMD CPU microcode updates
cpu-microcode-intel-20240531   Intel CPU microcode updates
cpu-microcode-rc-1.0_2         RC script for CPU microcode updates
root@router:~ # pkg query %v cpu-microcode-amd
root@router:~ #

And indeed in BIOS itself there is this line:

Ucode Patch Version                        810100B

So that answers that :D

I got my hands on the DEC750 I have and resetted it to stock, cold booted, and indeed with at the least latest bios for the moment (version 30), it seems to already have the latest microcode:

Code Select Expand

root@OPNsense:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
CPU0: local APIC error 0x80
Microcode patch level: 0x810100b

Sorry! It did _not_ print anything about microcode in latest reboot. I grepped microcode and it showed the 2 earlier boots... last one does NOT have microcode in it, ofcourse it didn't show up with grepping...

Code Select Expand

VT(vga): resolution 640x480
CPU: AMD Ryzen Embedded V1500B                       (2196.03-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)


ADD: Microcode patch level: 0x810100b still, so there is that..

I tried with stock kernel, it indeed seems that microcode has been stayed the same, again hard problem is that it is "only" warm reboot...

Code Select Expand

olmari@router:~ $ uname -a
FreeBSD router.huutoniemi 14.1-RELEASE-p2 FreeBSD 14.1-RELEASE-p2 stable/24.7-n267758-4ad7ad40bc77 SMP amd64

root@router:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b

root@router:~ # dmesg | grep microcode -A 20 -B 1
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2195.94-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222998528 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
--
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2196.06-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222994432 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.


Hmm, even if I change cpu_microcode_load="NO" I get same results... So the "a" might be true after all..

Code Select Expand

root@router:~ # dmesg | grep microcode -A 20 -B 1
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2195.94-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222998528 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
--
VT(vga): resolution 640x480
CPU microcode: no matching update found
CPU: AMD Ryzen Embedded V1500B                       (2196.06-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8222994432 (7842 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INSYDE EDK2    >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.

root@router:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b


EDIT: Though again "only" warm boot, as devide is still in remote location, so.. meh.. :D

a) In theory yes, in practice I'd say no because before any microcode update attempt of any kind I don't see the microcode patch level message at all, I suppose I could try to confirm this 100%

b) That is why I now tested cold power cycle in the last message (with driving to on premises :P ). same result.

c) correct lines in tunables -> loader, as per my earlier post..