Adding Speed Parameters to X550 Config

[spetrillo]

Here is the info right from the Amazon listing:

10Gb PCI-E NIC Network Card for Intel X550-T2,2.5GbE/5GbE/10GbE Copper Dual RJ45 Ports,with Intel ELX550AT2 Chip,10GbE PCI Express 3.0 X4 Ethernet Adapter Support Windows/Linux/VMware

So it seems the card is called a X550-T2 but the chipset is AT2. Did i read that correctly?

[BrandyWine]

So it seems the card is called a X550-T2 but the chipset is AT2. Did i read that correctly?

Let me explain.
1) You have it in reverse. Chipset is the x550, T AT2 BT2 are the form-factors.
2) Still need "dmesg |grep ix" to see how the system is identifying your x550. It's been seen before, weird device ID's are reported.
3) What switch make/model do you have?
4) I missed it earlier from Intel doc (page 24) in post #28, x550 is 802.3bz compliant. To get 2.5 5G both sides needs to be 802.3bz. It's very possible a "bad" cable causes link to downshift. Is your cable 802.3bz certified? A decent Cat5e cable should support 2.5G w/o issue.

[BrandyWine]

I'm not aware of any cable testing or training as part of any Ethernet link establishment - the link is simply negotiated to the highest common advertised speed

Not the case in 802.3bz, as long as the devices are fully 802.3bz compliant. "NBASE-T Downshift" is the coding name for the learning. It's a method to detect if the cable is causing packet issues, the coding can then do a speed downshift in an effort to remedy the packet issue, this helps keep data moving err free but sacrificing some speed.

A good read (https://archive.nbaset.ethernetalliance.org/wp-content/uploads/2017/05/NBASET-Downshift-WP-1217.pdf)
Alien noise in home environments can be an issue just like a bundle of Cat5e/6 in structured wiring in buildings. More rare in the home though.

however, the quality of the cable, how
cables are bundled together, or "alien noise"

IEEE 802.3bz Auto-Negotiation Detail
The 2.5GBASE-T and 5GBASE-T standards use IEEE 802.3bz
Auto-Negotiation, specified in Clause 28, Clause 126.6, Annex
28B, Annex 28C, and Annex 28D.
As part of the link-training sequence,

[pfry]

Not the case in 802.3bz, as long as the devices are fully 802.3bz compliant. "NBASE-T Downshift" is the coding name for the learning.[...]

I had not seen that. Heh. Learn something new every day. I wonder what the implementation rate is. There is some material flying around on the Internet that claims "wide implementation". Perhaps it's been hanging around in the background.

[BrandyWine]

Yet another update.
According to the driver code, you cannot use "dev.ix.<X>.advertise_speed" (well you can, but driver code disables based on phy).
The code is only checking for "phy.autoneg_advertised == 0", so even if per iface is set and the phy is 0, the driver will disable 2.5 5 autoneg capable bits.
More confusing in, Intel docs for x550 explicitly state 2.5 5 bits are set disabled in nvm, so by default those are out, so you must turn them on.

I would recommend using "phy.autoneg_advertised" tunable for best opportunity to get good 2.5 link. Or, set it to a non-zero and then also use the dev per nic setting. I wonder if phy setting overrides dev setting or vice-versa when they don't match?

But perhaps a catch-22. If the phy setting is set to 22, that part of ix code won't match the 'if' statement (so it does nothing), but where then does the ix driver code enable that 22 on the nics, because according to Intel docs for x550, the nvm has by default 2.5 and 5 autoneg capable bits set to disable. So now need to look into other parts of the driver code to see how the driver will set the bits to 'enable' when dev or phy sysctl setting is set to non-zero.

Code Select Expand

[b]if (hw->mac.type == ixgbe_mac_X550 &&
            hw->phy.autoneg_advertised == 0)[/b] {
            /*
             * 2.5G and 5G autonegotiation speeds on X550
             * are disabled by default due to reported
             * interoperability issues with some switches.
             *
             * The second condition checks if any operations
             * involving setting autonegotiation speeds have
             * been performed prior to this ixgbe_config_link()
             * call.
             *
             * If hw->phy.autoneg_advertised does not
             * equal 0, this means that the user might have
             * set autonegotiation speeds via the sysctl
             * before bringing the interface up. In this
             * case, we should not disable 2.5G and 5G
             * since that speeds might be selected by the
             * user.
             *
             * Otherwise (i.e. if hw->phy.autoneg_advertised
             * is set to 0), it is the first time we set
             * autonegotiation preferences and the default
             * set of speeds should exclude 2.5G and 5G.
             */
            autoneg &= ~(IXGBE_LINK_SPEED_2_5GB_FULL |
                IXGBE_LINK_SPEED_5GB_FULL);