Messages

I have an Deciso DEC750 with I-225, originally it had FW version that wasn't listed, I managed to update FW to version 1.89 1mb variant, and I do have backup from original(s). One thing I wonder that backups from all 3 I-225 interfaces are 2mb even if the 1mb v1.89 firmware went in just perfectly initially.

Some background: Now the one visual issue is that interface led are just static green led and nothing else, it doesn't affect the apparent working of interface otherwise. But now the DEC750 reboots between 1-3 days, and I'm not even sure this FW mangling is the rootcause, but:

I can not downgrade / change firmware to any version, it always fails even when trying to push the original backup back... Here is some log of _a_ attempt:

```
root@router:~/tik # cat nvm.log
Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.43.20.0
Copyright(C) 2013 - 2025 Intel Corporation.

./nvmupdate64e -b -l nvm.log -f -u -c 68.cfg

Config file read.
Inventory
[00:002:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Alternate MAC address is not set.
   Flash inventory started.
   Shadow RAM inventory started.
Warning: Can't preserve PBA. Device PBA size is incorrect.
   Shadow RAM inventory finished.
   Flash inventory finished.
[00:003:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Alternate MAC address is not set.
   Flash inventory started.
   Shadow RAM inventory started.
Warning: Can't preserve PBA. Device PBA size is incorrect.
   Shadow RAM inventory finished.
   Flash inventory finished.
[00:004:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Alternate MAC address is not set.
   Flash inventory started.
   Shadow RAM inventory started.
Warning: Can't preserve PBA. Device PBA size is incorrect.
   Shadow RAM inventory finished.
   Flash inventory finished.
Update
[00:002:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Creating backup images in directory: F490EA00B95F.
   Backup images created.
   Flash update started.
Error:      Flash update failed.
   Device update failed.
[00:003:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Creating backup images in directory: F490EA00B960.
   Backup images created.
   Flash update started.
Error:      Flash update failed.
   Device update failed.
[00:004:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Creating backup images in directory: F490EA00B961.
   Backup images created.
   Flash update started.
Error:      Flash update failed.
   Device update failed.
Update security revisions
[00:002:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Skipping update minimum security revisions.
[00:003:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Skipping update minimum security revisions.
[00:004:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Skipping update minimum security revisions.
Update VPD with VPD template
[00:002:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Skipping VPD update with VPD template.
[00:003:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Skipping VPD update with VPD template.
[00:004:00:00]: Intel(R) Ethernet Controller (3) I225-V
   Skipping VPD update with VPD template.
```

So I suppose this is sort of "what could I try next?" -question :)

Shameless cross-post from https://github.com/BillyCurtis/Intel-I226-V-NVM-Firmware/issues/3

If anyone is interested, I did successfully upgrade the i226-V's in a DEC750 to 2.32.

I used the 1MB bin file from BrandyWine.

I tried to search for any sort of changelog or release notes for the firmware, but could not find... I'm now thingking of upgrading too as they rarely put engineering effort for new firmware unless there is reason to...

I've in the middle of https://forum.opnsense.org/index.php?topic=48695.30 (post #39) and I seem to have I225-V in my DEC750 instead I226(?)
[1] igc0: <Intel(R) Ethernet Controller I225-V> mem 0xd0a00000-0xd0afffff,0xd0b00000-0xd0b03fff at device 0.0 on pci2
[1] igc0: EEPROM V1.82-0 eTrack 0x80000266

EDIT: Apparently filling in the nvm.cfg with latest 1M file for I225 made it work, so now I have 1.89 of I225-v Firmwares in all 3 wired NICs in my DEC750.

The one thing I've always wondered that does the CARP IP's really need to be on same subnet or could those be on "arbitrary" but lone network segment of, say 192.168.100.1/30 and .2...

I know at my previous job some UXG firewall, also freebsd based, got away without needing the interface IP's to be on same subnet, but sadly I did not investigate how it was achieved...

I don't think the routers own NVMe would be best target nor source.. Or generally any single NVMe for that matter... If you wanna test the networking capabilities of the router, do it with, say, 2 hosts using SPF+ each and whatever perf test you manage to do that just throws data to /dev/null in practice... Otherwise you're bot testing networking performance quite at all...

I see the 24.7.6 having early loading landed in it, thank you! Only thing now missing from the "final test" for my devices is the actual microcode release by AMD. xD

Indeed only thing at the moment preventing me to test happy-path of Amd early loading is lack of more fresh microcode, indeed otherwise I'm happy with what we have had so far (the check and "no patch to update" as DEC750 already has the current latest =)

Well, I know this is/was an long shot... Many, especially low-end consumer routers, has every led behind GPIO practically, whole another thing is that is it ever exposed with OEM firmware, with Openwrt it is usually available in those =)

Then another thing is that indeed sometimes NIC/PHY drivers/firmware/xxx has undocumented features where led control is one of them...

Ofcourse I know there exists MacGyver solutions with Jesus-tape and pineapples and tree sap, but software control would have been awesome ;P

For f**k sakes...

But this also looks like the Amd early loading system itself works, at least the "no updates found" part. Like Fitch Franco (at forum side :D ) said earlier, I think this is now mostly waiting game for updated suitable microcode package for the actual code loading part to happen on this system :)

I have two DEC750 and one DEC2752 where one of the 750s is especialyl at parade place at domestic enviroment. I'd so wish there would be an way to basically turn all lights/leds off... Optionally there could be some "one light blinks every minute to show it is on" if such is desired, but generally if I could jsut turn of the blinkenlights and sleep better at nights would be awesome. :P

Code Select Expand

root@router:~ # opnsense-revert -z cpu-microcode-amd
Package 'cpu-microcode-amd' is not installed
root@router:~ # pkg query %v cpu-microcode-amd
root@router:~ # pkg install cpu-microcode-amd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
cpu-microcode-amd: 20240116
cpu-microcode-rc: 1.0_2

Number of packages to be installed: 2

60 KiB to be downloaded.

Proceed with this action? [y/N]:
root@router:~ # pkg install cpu-microcode-amd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
cpu-microcode-amd: 20240116
cpu-microcode-rc: 1.0_2

Number of packages to be installed: 2

60 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/2] Fetching cpu-microcode-rc-1.0_2.pkg: 100%    3 KiB   2.6kB/s    00:01   
[2/2] Fetching cpu-microcode-amd-20240116.pkg: 100%   58 KiB  59.4kB/s    00:01   
Checking integrity... done (0 conflicting)
[1/2] Installing cpu-microcode-rc-1.0_2...
[1/2] Extracting cpu-microcode-rc-1.0_2: 100%
[2/2] Installing cpu-microcode-amd-20240116...
[2/2] Extracting cpu-microcode-amd-20240116: 100%
=====
Message from cpu-microcode-rc-1.0_2:

--
This port includes an RC script, which is one of two methods to update
the CPU microcode on a FreeBSD system.

1. Early loading.
   This method does not use the RC script included here.
   This is the preferred method, because it ensures that any CPU features
   added or removed by a microcode update are visible to the kernel by
   applying the update before the kernel performs CPU feature detection.

   To enable updates using early loading, add the following lines to
   /boot/loader.conf:

   cpu_microcode_load="YES"

   and the appropriate one of these lines:

   cpu_microcode_name="/boot/firmware/intel-ucode.bin"
   cpu_microcode_name="/boot/firmware/amd-ucode.bin"

   The microcode update will be loaded when the system is rebooted.

   AMD systems running FreeBSD prior to 2024-02-22 snapshot
   34467bd76 only support late loading.


2. Late loading.
   This method, which does use the RC script included here, is enabled by
   adding the following line to /etc/rc.conf:

   microcode_update_enable="YES"

   The microcode update is then applied upon reboot or when the microcode
   update service is run via:

   # service microcode_update start

   If the CPU requires a microcode update, a console message such as the
   following will appear:

   Updating CPU Microcode...
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl0 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl2 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl4 from rev 0x17 to rev 0x22... done.
   /usr/local/share/cpucontrol/m32306c3_00000022.fw: updating cpu /dev/cpuctl6 from rev 0x17 to rev 0x22... done.
   Done.

It is safe to enable both methods.
=====
Message from cpu-microcode-amd-20240116:

--
Refer to the cpu-microcode-rc installation notes to enable AMD microcode
updates.
root@router:~ # opnsense-revert -z cpu-microcode-amd
Fetching cpu-microcode-amd.pkg: ... done
Verifying signature with trusted certificate pkg.opnsense.org.20240611... done
cpu-microcode-amd-20240116: already unlocked
Installing cpu-microcode-amd-20240810...
package cpu-microcode-amd is already installed, forced install
Extracting cpu-microcode-amd-20240810: 100%
=====
Message from cpu-microcode-amd-20240810:

--
Refer to the cpu-microcode-rc installation notes to enable AMD microcode
updates.
root@router:~ # pkg query %v cpu-microcode-amd
20240810

Dmesg: CPU microcode: no matching update found

root@router:~ # kldload -q cpuctl; x86info -a | fgrep -i microcode
Microcode patch level: 0x810100b

So, either something does not load correctly, or for Ryzen v1500b there is no microcode update in this round. My bet is in the latter by preliminary looks.

Maybe I'm dense, but on 24.7.1 (stock and amd-early kernel) I get no updated cpu-microcode-amd package..

Code Select Expand

root@router:~ # pkg update
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
root@router:~ # pkg search cpu-microcode
cpu-microcode-1.0_1            Meta-package for CPU microcode updates
cpu-microcode-amd-20240116     AMD CPU microcode updates
cpu-microcode-intel-20240531   Intel CPU microcode updates
cpu-microcode-rc-1.0_2         RC script for CPU microcode updates
root@router:~ # pkg query %v cpu-microcode-amd
root@router:~ #

And indeed in BIOS itself there is this line:

Ucode Patch Version                        810100B

So that answers that :D

I got my hands on the DEC750 I have and resetted it to stock, cold booted, and indeed with at the least latest bios for the moment (version 30), it seems to already have the latest microcode:

Code Select Expand

root@OPNsense:~ # kldload -q cpuctl ; x86info -a | fgrep -i microcode
CPU0: local APIC error 0x80
Microcode patch level: 0x810100b

Sorry! It did _not_ print anything about microcode in latest reboot. I grepped microcode and it showed the 2 earlier boots... last one does NOT have microcode in it, ofcourse it didn't show up with grepping...

Code Select Expand

VT(vga): resolution 640x480
CPU: AMD Ryzen Embedded V1500B                       (2196.03-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f10  Family=0x17  Model=0x11  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)


ADD: Microcode patch level: 0x810100b still, so there is that..