Intel X520-DA2 producing Errors IN

[Chrome]

Hi there,

Need some help determining what exactly the cause problem is and the solution.

The issue is, when I use a 10G SFP+ connection to my Aurba S2500 for my LAN connection, I get an ever increasing amount of ERRORS IN:




Environment:
Fresh OPNsense install and update to version 22.7.6 (was happening with 22.7.5)
Motherboard - Supermicro X10SLL-F
Tried two different NICs - Genuine Intel X520-DA2 (brand new) and Supermicro AOC-STGN-I2S Rev.2 (X520-DA2 variant), both produce the "ERRORS IN".


I have tried different SFP+s, mixed brands, same result. I have not tried a different cable, only one available now, waiting for a delivery with 2 more.


When I change the card to a Mellanox ConnectX-3 (same SFPs, same cable, same switch) - NO ERRORs


I am using the onboard em0 for the WAN connection (no errors).


Not sure what else is left to try?

[h4ck3r]

We think it may be related to Freebsd 13 when we switch from version 21.x to version 22.x. There are errors in intel x540-t2 card. No problem with intel I350-t rNDC cards :/

https://www.freebsd.org/releases/13.1R/hardware/ is there information and support for your current card here?

[Chrome]

The x520 card is supported under the IX driver.

The em0 and igbe NICs (both onboard) work well, with no errors.

I have loaded Windows and Ubuntu on the same setup...and the card seems to work (as it does in OPNsense), granted I don't know how to check if there are "ERRORS IN" on those two OSes...but my poking around didn't turn up anything negative.

So, I am wondering if there is an incompatibility between the X520 chipset, Freebsd and my Aruba S2500 (don't have another switch to check it against).

Or

Just between FreeBSD and the card.... but the card is a popular one for Freebsders, and seem well supported/documented.

[Chrome]

Here's the output of a netstat -i log from OPNsense:




Here's the output of a netstat -i log form Ubuntu (enp2s0f0):

[Chrome]

Booted with FreeBSD 13.1 DVD here's the output of netstat -i log:

[ReDaLeRt]

Have you tried the following settings?

Interfaces -> Settings -> Disable hardware checksum offload
Interfaces -> Settings -> Disable hardware TCP segmentation offload
Interfaces -> Settings -> Disable hardware large receive offload
Interfaces -> Settings -> Disable VLAN Hardware Filtering

[Chrome]

Those settings stayed as default... but here's what they look like:

[franco]

Possibly... https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266048 ... change by Intel...

EDIT: I prodded a FreeBSD contact about the situation.. a pragmatic fix seems to exist in DPDK:

https://git.dpdk.org/dpdk/commit/?id=2ee14c8905e9


Cheers,
Franco

[Chrome]

Thanks Franco.

I believe my issue and the bug you linked to Freebsd are totally related.

I added my comment to the Freebsd link for a cross reference.

I review the DPDK you mentioned... the fix seems to be there.

Hopefully, its included in the next release.

I am using the Mellanox Connectx-3 for now... may switch to the Intel card when this bug gets fixed.

[franco]

To be honest, the bug is a statistics bug -- not more and not less. This does not affect operation of the card as far as I can tell...

I decided to make a patch and will put it up for review if you can confirm the errors are no longer reported.

https://github.com/opnsense/src/commit/4a788be44e0395

You can try the kernel as follows:

# opnsense-update -zkr 22.7.5-ixgbe
# opnsense-shell reboot


Cheers,
Franco

[seed]

@franco

is it possible that this also is a statistics bug:

https://github.com/opnsense/src/issues/74
https://forum.opnsense.org/index.php?topic=15685.msg71760#msg71760

[franco]

Supposedly this output error thing only happens with VLAN interfaces with Netmap grabbing packets as well? Or at least that was one instance where it happens (unrelated to this input error which is clear Intel errata).


Cheers,
Franco

[Chrome]

To be honest, the bug is a statistics bug -- not more and not less. This does not affect operation of the card as far as I can tell...

I decided to make a patch and will put it up for review if you can confirm the errors are no longer reported.

https://github.com/opnsense/src/commit/4a788be44e0395

You can try the kernel as follows:

# opnsense-update -zkr 22.7.5-ixgbe
# opnsense-shell reboot


Cheers,
Franco

@Franco - That worked!! Thank you! Here's a snippet:



I agree, its a statistics bug ... nothing more than that. I don't believe it affects performance of the card in anyway.

[franco]

Thanks for confirming. FreeBSD committer proposed a similar patch in the bug report... confident this will be included shortly.


Cheers,
Franco