Any issues with i210 / i211 & i350 on OPNSense 20.x ?

[packetmaster]

Hi all,

Has anyone experienced any of the issues with Intel NICs in the latest OPNSense (20.x) outlined in this thread?

https://forum.opnsense.org/index.php?topic=5511.0

I am looking to implement a new firewall set up with latest version of OPNSense.  I am looking at hardware from guys like Jetway, Qotom, Protectli with newer Kaby Lake processors.  Jetway and Qotom use the i210 / i211 and i350 NICs while the Protectli uses older 82583V.

I would think the newer NICs would be better but that thread has me worried.  That said, seems like Amazon reviews have a lot of people using PFSense or OPNSense on the Qotom which lead me to believe the i210 / i350 issues have been sorted out, or Wener (poster in the thread above) was running into issues with the particular Jetway hardware.

[Superduke]

FWIW...I'm using 82575 based cards and no issues so far....that said, just built my box and just recently started using OPN so who knows at this point.....wanted better/newer cards, but couldn't force myself to spend the extra money on the newer ones.....maybe that'll come back to bite me   

[mimugmail]

i211 doesn't work really stable .. rest is ok

[PWCDC]

Using a QOTOM with a i211. Experiencing daily drops in the interface assigned to LAN. Logs are unclear. I'm not the only one. There appear to be a couple of threads about it. The i211 appears to be the common denominator.

I think its Opnsense related, since the same hardware worked perfectly on PFsense.

[franco]

> I think its Opnsense related, since the same hardware worked perfectly on PFsense.
Modify message

Qualified statements please: Which OPNsense and pfSense versions are we comparing? You are aware the two have different OS versions in production releases?


Cheers,
Franco

[PWCDC]

> I think its Opnsense related, since the same hardware worked perfectly on PFsense.
Modify message

Qualified statements please: Which OPNsense and pfSense versions are we comparing? You are aware the two have different OS versions in production releases?


Cheers,
Franco

Fair comment. The pfSense version was the latest stable. (2.4.5) pfSense doesn't post updates very often. Their stable branch is still on BSD 11 I think. They were on the verge of 2.5, but i'm not sure if that has been released yet. I lost interest in pfSense after finding out more about the organizations history.

Every Opnsense version I've tried has been BSD12.x. I ran 20.7 for a bit over a week then upgraded to 21.1.

The problem may lie with BSD12.

[franco]

I think so too. I have some more things to merge from Intel from later stable patches after 12.1 was finalised anyway and look for more, but I need a bit of time...

https://github.com/opnsense/src/issues/100


Cheers,
Franco

[astromeier]

Nice to read, franco, that you are working on the intel issues!
Can we hopefully forget the issues with the common intel i211 NICs in the next months?
Would be great!

[PWCDC]

I know there are a lot of priorities, but this seems like it should be a higher one.

This "glitch" effectively means the newer OPNSense versions cannot work reliability on many of the QOTOM/Protectli mini-computers that you can get off amazon/aliexpress that are very popular to run with PFSense/OPNSense. A majority of them use the i211 chipset.

I'm actually a little surprised more people aren't remarking on this. It makes me feel as though there is something unusual about my configuration. I suppose its possible most home/SOHO users may not notice dropouts several times a day.

[thowe]

Some of the PC Engines APU2/3/4 also have the i211 onboard. (Some do have the i210, which seems fine).

Since these APU boards are widely used: Has no owner noticed yet? (I have just APUs with i210.)

[Patrick M. Hausen]

I am running an apu4d4 with the 211 chipset and don't experience any issues.

2 of the 4 interfaces are combined in a lagg, plugged into a Cisco switch with LACP, on top of that I run 2 VLANs for LAN and OPT1. The system does not have a WAN interface.

Sorry, sort of - just humming along.

[PWCDC]

To be honest, I probably wouldn't have noticed if we didn't have people working from home due to Covid. Lots of video conference calls getting interrupted. According to the logs, the dropouts happen 3 or 4 times a day, seemingly at random (no idea what triggers it). It flipflops for about 10-15 minutes then comes back up.

A lot of the time, this happens overnight, or during work hours. It is very possible someone could be having this issue and simply not noticing it. 

If you look through your system logs, you will see something like:

igb0: link state changed to DOWN

several times per day.

Then up and down a few more times. If this happened at a time when you weren't actively using your internet connection, you may not have noticed, or assumed it was a one-off.

[Patrick M. Hausen]

Nope 

Code: [Select]

root@opnsense:/var/log # uptime
 5:17PM  up 5 days,  1:37, 1 user, load averages: 0.50, 0.43, 0.38
root@opnsense:/var/log # dmesg | grep 'link state'
root@opnsense:/var/log # grep 'link state' *

I used to have minor problems with 20.7. Initially I bridged all 4 interfaces of my apu4 to use the device as a small home switch - which JeGr despises but I find perfectly reasonable. Sometimes after config changes I would lose connectivity to my desk where my Mac is connected via a Thunderbolt docking station with Ethernet and only a reboot of OPNsense would bring it back.

Since I bought a Cisco 2960-L 16-port switch and connected all servers (2x TrueNAS, 1x OPNsense) to that via LACP and then access ports to other systems in the apartment I have not had any problems.

So I *guess* the device on the other end can play a role, too.

[PWCDC]

That's kind of what I was afraid of.

I've replaced the cable, switch, and isolated the physical network down to a few devices and it has not resolved the dropouts. I've even changed the port (the device has 4).

When I started seeing threads about the i211 pop up, I assumed that was it. Especially since it wasn't happening on older BSD versions.

I'm starting to wonder if I have a bad QOTOM unit. Given the sketchy nature of the manufacturing on these things, I suppose it's possible.

[Ricardo]

1) I simply dont understand why pcengines switched away from i210 and use i211 in their higher numbered APUs (APU4, APU5, APU6, these are not even listed on pcengines.ch, the secrecy inner workings of this swiss company is  confusing as hell to me). As you all should be aware i211 is inferior to i210 (in contrast what the model number may suggest, bigger number != better product in this case). i210 supports 4 TX&RX queues per port, i211 has only 2 TX&RX queues per port. Is significant difference, because Receive Side Scaling can distribute incoming (TX) packets between the CPU cores, if there are as many queues as CPU cores. Otherwise only 1 or max. 2 cores will get all the processing work of the incoming packets, while the rest of the CPU cores will sit idle nothing to do. That is a real life issue above 100Mbit speed, as the APU 1Ghz AMD embedded CPU has very weak single core performance, and all 4 cores would be needed to process packets at 1Gbit, which is more or less impossible with the i211 nic in many of the APUs.
Pcengines does not seem to work on a future improvement on this dead horse APU design to put higher clocked AMD SoC in to the PCB, and replace i2xx with i3xx that already has 8 TX & RX queues per port.