Messages

1

Hardware and Performance / Re: [NIC] X550-AT2 802.3az/Energy-efficient Ethernet (EEE)

« on: February 27, 2022, 10:04:51 am »

Hi

With the newest software, it does not seem to support it (neither on my linux box nor the OpnSense).

Code: [Select]

media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
        status: active
        supported media:
                media autoselect
                media 100baseTX
                media 1000baseT
                media 10Gbase-T
I know that the *hardware* supports it, though - it's just the drivers that don't. It was apparently removed at some point due to "issues with negotiations". These card/driver-combos are really in a sorry state.
Maybe the Intel drivers do support it, but I cannot really test that without risking to crash the production systems

If your footprint allows it you could always strap a small 40/60MM FAN to the card to help dissipate heat. I know this isn't ideal but I've done that in the past with other devices than ran warm.

Well, yes. If it was a cheapo realtek Chip for 10$ I could accept that. This is an IC that costs 150$. And removing generated heat is fine, but it still is wasted if you could get away with *industry-standard* stuff that was even advertised at one point in 2016(!).
Well, anyways, I grabbed my pitchfork over at the folks of Intel to get it cleared out. I'll keep you posted; if somebody wants to join, feel free.

https://community.intel.com/t5/Ethernet-Products/On-the-state-of-802-3az-with-Intel-X550/m-p/1364016/

2

Hardware and Performance / [NIC] X550-AT2 802.3az/Energy-efficient Ethernet (EEE)

« on: February 22, 2022, 04:32:08 pm »

Hi

I'm successfully running a couple of machines using the X550-AT2 NIC Chip. Currently, I'm using this Board for an integrated Setup:
https://www.asrockrack.com/general/productdetail.asp?Model=EPYC3101D4I-2T#Specifications

All is well, but the NIC gets extremely hot (90° C). Yes, 10GBase-T is not very efficient and according to Intel, uses up to 13 Watts of power that all has to go somewhere.
However, there should be a solution with 802.3az - The energy-efficient ethernet Standard. My switch does support it.
For some reason, even though the datasheet explicitly lists the registers for the X550, support got killed at some point due to rather esoteric NICs/Phys (55[2,3]):
https://reviews.freebsd.org/D21673

Since then, "thing's just gone, yo":
https://github.com/freebsd/freebsd-src/blob/d5c0a7b6d3923d2a6967810d0aa3e148a39351c1/sys/dev/ixgbe/ixgbe_x550.c#L66

The eee_status / eee_enabled flags are gone from dev.ix.<num>.

However, even though Intel is not officially supporting EEE anymore on X550 (only on x710+), the registers in silicon probably did not yet vanish.
Is there a way to force the driver to pretty-please allow for EEE to enable? After all, the X550 is a rather often used cheap 10G chip and the overall powersavings could be substantial.

Best
cwriter

3

Hardware and Performance / [NIC] Intel X550 SR-IOV with VLAN

« on: February 28, 2021, 12:30:38 pm »

Hi

I have a somewhat special use case:
I need to split a single port of an Intel X550-T2 into 5 different subinterfaces, 4 of which are carrying a VLAN tag with all different MAC addresses.

There have been a couple of hickups on the ride. It's possible to create VLAN-subinterfaces on ix0, yes. But then, I cannot have unique MACs (FreeBSD sets one MAC for parent and all subinterfaces).
I can get 2 nets going by creating a bridge, which allows getting 2 different MACs. But since you cannot have multiple bridges, this does also not work.

Then I remembered that the X550-T2 allows splitting one Physical Device into multiple Virtual Functions by using SR-IOV. I got this to work with different MACs by using iovctl to split the Device. After splitting, the parent (ix0) is unresponsive, but the created interfaces (ixv0 through ixv4) work fine.
I did the splitting by compiling the ix and ixv drivers, putting

Code: [Select]

if_ix_updated_load="YES"
if_ixv_updated_load="YES"
into /boot/loader.conf.local, copying the compiled drivers (with SRIOV_ENABLE=1) into /boot/modules/ and adding a syshook script to run iovctl.

However... When creating a VLAN-Subinterface, ix0 and all (VF) children start dropping packets and do not respond anymore.

Now: I know that SR-IOV was primarily intended for passing them to a guest vm, which I am not doing. I'm doing this only as a workaround to get unique MAC addresses.
I suspect that there are issues due to loading the ixv driver which should not allow setting VLANs nor enabling promiscous mode due to security concerns. However, when creating a VLAN subinterface on ixv1, ixv0 drops connection. So I suspect some kind of underlying bug either in the driver or in the error handling when creating a subinterface.

Has someone else experienced issues with X550, SR-IOV and VLANs? I've found conflicting information on the web about this, e.g.: https://netbsd.itsx.net/freebsd+bhyve+sriov.html


Thanks