Messages

@BrandyWine
Sorry i meant Mb = Mbit.

I have found similar reports on the net regarding slow speeds when ASPM is enabled:
https://www.reddit.com/r/intelnuc/comments/1951rgk/very_slow_speeds_with_i226v_on_intel_nuc13anbi5/
https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20241202/045288.html

When i am reading this correctly, it seems a driver issue which seems to fixed already in windows driver but not linux?

@BrandyWine With "link" i meant the PCIe link speed. Sorry but i dont get it. ASPM is designed to lower or cut off the PCIe link when the device is idle. So when it's not idle the whole PCIe link bandwidth should be available, hence also the 2,5GBE ethernet speed.

A few months back i had an realtek RTL8125B where ASPM was working as it should, up to the point where unraid (driver or linux kernel) removed ASPM support.
You had to use an own driver for it to get i working again, but ASPM was working fine with always 2,5 GBE speeds.

There is clearly an issue in the mainboard BIOS, card firmware, driver or linux kernel when it's stuck @ ~600 MB/s with ASPM on.

I will leave it off for now for the card.

Sure 2,5 GB/a will draw more power than GB speeds, but i think this could be handled by firmware and/or driver.

Like so:
If ASPM mode enabled and pci-e link is in an low power mode while bandwidth hits an certain threshold for an specified amount of time, card should requesting system to put link out of low power mode so 2,5 GB/s will be reached. If bandwidth is below an certain threshold, low power mode will be activated again.

This way we can have our systems/CPU's can reach higher C states and safe more juice out of our walls.

There's a boot kernel cmdline argument to disable ASPM. Almost every *nix kernel has it. 

And yep, I think it was mentioned in this thread, there are various flashing utils depending on type of OS.

I don't think REPLACES is a match tag like you think. IIRC it tells the flash util that the flashing will proceed if the supplied ETID is matched. You can target the nic by MAC, or without any targeting at all, so the util provides some safeguards when you want to target just a subset of all 226's, etc. As example, if some 226's were already at 2.32 but others were older, you could target just the older ETID's. Kinda same thing using MAC as an attribute for target. The utils need updating, they are currently crude and convoluted, and the docs are not clear. When the nic device has area for OROM, then things get even more complicated.

Glad to hear you got your slackware 226's updated.

Thx for your detailed answer.
I don't want to disable ASPM all together.
With my approach I can have ASPM enabled for all other devices and just disable it for the i226-B, with this I am reaching C3 on my system an save a little bit of juice.

P.S.: when enabling ASPM for the i226-V card I am reaching C6 and saving another 2-3 Watts. Sadly with just 600 MB/s

[dmesg]

You should keep ASPM disabled.

You listed command as "dmsg", should that be "dmesg |grep EEPROM"

unraid? Hmmmmm, slackware.
I guess 1st step will be to understand that customized distro. Does igc show up at all?
run lspci -vv and lsmod -v , find your i226v and driver module for it. It's possible the unraid folks compiled the Intel igc source to a different name if igc does not show up.

After that, I would use the flash procedures in the thread and flash them to v2.32. Highly probable they need updating. Use the correct flash utility for linux. I suspect after updating the 226's your issues should go away, but still disable ASPM because 2.5Gb needs more power to operate and if ASPM enters a lower power state it can cause Tx/Rx issues if the nic does not re-negotiate link down to 1Gb. The driver should handle this scenario, but it's hard to diagnose because if the the device does go into lower power state but the actual link info stays at 2.5G, which it can't do because of the power state it's in.

Btw, you can use the flash utility to just probe the hardware for info.

Since this is an OPNsense forum I will suggest you can get much more info for unraid (slackware) from linuxquestions org

Sorry i meant "dmesg |grep EEPROM".
Yes, the log has some igc stuff but the etrack version was missing.

I finally upgraded my rather old 2.13 to 2.32.
The key in unraid was to use the linux I225 nvmupdate64e binary from the "Intel® Ethernet Adapter Complete Driver Pack" from here: https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html

The other process was the same:

Inventory before (XXXXXXXXXXXX is your MAC):

Code Select Expand

./nvmupdate64e -i  -l

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.41.3.1
Copyright(C) 2013 - 2024 Intel Corporation.

Config file will not be read.
Inventory
[00:001:00:00]: Intel(R) Ethernet Controller I226-V
        Alternate MAC address is not set.
        Flash inventory started.
        Shadow RAM inventory started.
        Shadow RAM inventory finished.
        Flash inventory finished.
[00:001:00:00]: Intel(R) Ethernet Controller I226-V
        Vendor                 : 8086
        Device                 : 125C
        Subvendor              : 8086
        Subdevice              : 0000
        Revision               : 4
        LAN MAC                : XXXXXXXXXXXX
        Alt MAC                : 000000000000
        SAN MAC                : 000000000000
        ETrackId               : 80000286
        SerialNumber           : XXXXXXXXXXXXXXXX
        NVM Version            : 2.19(2.13)
        PBA                    : G23456-000
        VPD status             : Not set
        VPD size               : 0
        NVM update             : No config file entry
          checksum             : Valid


modified/checked nvm.cfg (pay attention to the REPLACES! Which must be the old ETrackId id):
But laso check all other entries like VENDOR, ....

Code Select Expand

CURRENT FAMILY: 1.0.0
CONFIG VERSION: 1.20.0

; NIC device
BEGIN DEVICE
DEVICENAME: Intel(R) Ethernet Controller I226-V
VENDOR: 8086
DEVICE: 125C
SUBVENDOR: 8086
SUBDEVICE: 0000
NVM IMAGE: FXVL_125C_V_1MB_2.32.bin
EEPID: 80000425
RESET TYPE: REBOOT
REPLACES: 80000286
END DEVICE

Update, directly over SSH (XXXXXXXXXXXX is your MAC):

Code Select Expand

./nvmupdate64e -b -l nvm.log -m XXXXXXXXXXXX-f -u -c nvm.cfg

Inventory after (XXXXXXXXXXXX is your MAC):

Code Select Expand

./nvmupdate64e -i  -l

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.41.3.1
Copyright(C) 2013 - 2024 Intel Corporation.

Config file will not be read.
Inventory
[00:001:00:00]: Intel(R) Ethernet Controller I226-V
        Alternate MAC address is not set.
        Flash inventory started.
        Shadow RAM inventory started.
        Shadow RAM inventory finished.
        Flash inventory finished.
[00:001:00:00]: Intel(R) Ethernet Controller I226-V
        Vendor                 : 8086
        Device                 : 125C
        Subvendor              : 8086
        Subdevice              : 0000
        Revision               : 4
        LAN MAC                : XXXXXXXXXXXX
        Alt MAC                : 000000000000
        SAN MAC                : 000000000000
        ETrackId               : 80000425
        SerialNumber           : XXXXXXXXXXXXXXXX
        NVM Version            : 2.50(2.32)
        PBA                    : G23456-000
        VPD status             : Not set
        VPD size               : 0
        NVM update             : No config file entry
          checksum             : Valid

After that check nvm.log and reboot your device.
The log shoukld say succeed please reboot.

Sadly, my ASPM issue is not solved with the new version.

If i activate ASPM for the card:

Code Select Expand

echo 1 > /sys/bus/pci/devices/0000:01:00.0/link/l1_aspm

and run an iperf 3 after some idle time the bandwidth drops to ~600 MB/s...

Code Select Expand

iperf3 -c 192.168.0.5 -t 200
Connecting to host 192.168.0.5, port 5201
[  5] local 192.168.0.1 port 62245 connected to 192.168.0.5 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  67.9 MBytes   563 Mbits/sec
[  5]   1.01-2.00   sec  68.0 MBytes   576 Mbits/sec
[  5]   2.00-3.00   sec  67.2 MBytes   564 Mbits/sec
[  5]   3.00-4.00   sec  68.9 MBytes   577 Mbits/sec
[  5]   4.00-5.00   sec  69.1 MBytes   580 Mbits/sec
[  5]   5.00-6.01   sec  72.0 MBytes   600 Mbits/sec
[  5]   6.01-7.01   sec  68.0 MBytes   572 Mbits/sec
[  5]   7.01-8.01   sec  32.2 MBytes   269 Mbits/sec
[  5]   8.01-9.00   sec  41.8 MBytes   354 Mbits/sec
[  5]   9.00-10.01  sec  33.4 MBytes   278 Mbits/sec
[  5]  10.01-11.00  sec  49.4 MBytes   417 Mbits/sec
[  5]  11.00-12.01  sec  24.0 MBytes   200 Mbits/sec
[  5]  12.01-13.01  sec  33.1 MBytes   276 Mbits/sec
[  5]  13.01-14.01  sec  45.0 MBytes   378 Mbits/sec
[  5]  14.01-15.00  sec  31.5 MBytes   268 Mbits/sec
[  5]  15.00-16.01  sec  48.4 MBytes   400 Mbits/sec
[  5]  16.01-17.01  sec  28.4 MBytes   240 Mbits/sec
[  5]  17.01-18.00  sec  32.2 MBytes   271 Mbits/sec

So i will leave this in my boot script to disable ASPM for the card!

Code Select Expand

echo 0 > /sys/bus/pci/devices/0000:01:00.0/link/l1_aspm

iperf with ASPM disabled...

Code Select Expand

[  5]  37.01-38.01  sec   283 MBytes  2.37 Gbits/sec
[  5]  38.01-39.01  sec   285 MBytes  2.37 Gbits/sec
[  5]  39.01-40.01  sec   280 MBytes  2.37 Gbits/sec
[  5]  40.01-41.01  sec   283 MBytes  2.37 Gbits/sec
[  5]  41.01-42.01  sec   283 MBytes  2.37 Gbits/sec
[  5]  42.01-42.10  sec  26.9 MBytes  2.38 Gbits/sec

Anyway, thx for this really informative thread.
Now my card is up to date and i have to hope for an kernel/driver fix for my issue.

Hi guys.
FIRST THX FOR THIS NICE INFORMATIVE THREAD!

Running an I226-V in on an unraid server 7.2.0 (kernel 6.12.24) and have ASPM Issues.
When ASPM L0 is enabled for the card it just can handle ~300mBit/s after some idle time.
I have to disable ASPM for the card so it reaches 2,5 Gbit/s all the time.

Sadly the "dmsg | grep igc" doesnt show me the EEPROM firmware version.

@BrandyWine and others which have updated the firmware:

• Do you had this issue i am describing?
• Has the update solved the issue and is ASPM finally workling?

I know this is OPNSense forum, but would be nice if we can figure this out.

Here is an reference link to powertop on unraid: https://github.com/fenrus75/powertop/issues/143

thx

Hey guys.

First thanks a lot for your efforts von the NVM update and the ASPM issues with the I226.
I and others experiencing same issues in unraid, regarding ASPM, see here: https://github.com/fenrus75/powertop/issues/143

When we activating "PEG ASPM" in BIOS the NIC throughput of the I226-V drops significantly.
So we are currently disabling "PEG ASPM" in BIOS to get arround this issue.

Sure, the drawback is that the cpu does'nt reach higher c-states than C2/C3, or in other words, it draws more power from the wall.

When i got you right, you are trying to disable ASPM on the card all together.
Our goal (with powertop on unraid) is to enable it and the card should/could handle it, to save more energy.

Do you have any infos if that's a silicon issue and maybe is or can be fixed with an NVM update?
Maybe thats possible with disable a portion (L0, L1, ...) off the ASPM features on the card to preserve "some" ASPM feature and save energy.

Would be nice to hear your thought's.