Problems With Nics not been able to use

[Tuti1196]

Hello Friends

I have adquiere a Exinda 4062 hardware and succesfully intall opensense , now the hardware itsell shows the comes with 6 nics

But only the managment is the only one working the nics present as:

igb0  Intel(R) I210  (Copper)
igb1  Intel(R) I350  (Copper)
igb2  Intel(R) I350  (Copper)
igb3  Intel(R) I350  (Copper)
igb4  Intel(R) I354  (SGMII)
igb5  Intel(R) I354  (SGMII)
igb6  Intel(R) I354  (SGMII)

i understand that ports are like this: 3 Bridge Pairs, 1 Management

Any sugestions?

[pfry]

The number (2x3) is odd. If I had to guess, it's a Rangeley (Silvermont Atom), which normally have four i354 interfaces. If it has a hardware bypass, the three i350/354s could be in bypass mode. You could test this by plugging in two endpoints (any two you can test for connectivity; 1000BASE-T may be easier than 100BASE-TX or lower as you don't have to deal with polarity) (e.g. one test endpoint plugged into E, the other J). If it has a default bypass, it may be difficult to deactivate. Other than that, ya got me.

[BrandyWine]

lspci -vmm

[Patrick M. Hausen]

lspci -vmm

You probably mean

Code Select Expand

pciconf -lv


[BrandyWine]

Sorry, its in pciutils pkg, but pciconf is ok. Wondering what that looks like, etc.

[Tuti1196]

Sorry, its in pciutils pkg, but pciconf is ok. Wondering what that looks like, etc.

hostb0@pci0:0:0:0:      class=0x060000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f08 subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 SoC Transaction Router'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:1:0:       class=0x060400 rev=0x02 hdr=0x01 vendor=0x8086 device=0x1f10 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCIe Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib2@pci0:0:2:0:       class=0x060400 rev=0x02 hdr=0x01 vendor=0x8086 device=0x1f11 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCIe Root Port 2'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:3:0:       class=0x060400 rev=0x02 hdr=0x01 vendor=0x8086 device=0x1f12 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCIe Root Port 3'
    class      = bridge
    subclass   = PCI-PCI
pcib5@pci0:0:4:0:       class=0x060400 rev=0x02 hdr=0x01 vendor=0x8086 device=0x1f13 subvendor=0x8086 subdevice=0x8086
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCIe Root Port 4'
    class      = bridge
    subclass   = PCI-PCI
none0@pci0:0:11:0:      class=0x0b4000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f18 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 QAT'
    class      = processor
hostb1@pci0:0:14:0:     class=0x060000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f14 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 RAS'
    class      = bridge
    subclass   = HOST-PCI
none1@pci0:0:15:0:      class=0x080600 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f16 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 RCEC'
    class      = base peripheral
    subclass   = IOMMU
none2@pci0:0:19:0:      class=0x088000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f15 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 SMBus 2.0'
    class      = base peripheral
igb5@pci0:0:20:0:       class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1f41 subvendor=0x15d9 subdevice=0x1f41
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
igb6@pci0:0:20:1:       class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1f41 subvendor=0x15d9 subdevice=0x1f41
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
igb7@pci0:0:20:2:       class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1f41 subvendor=0x15d9 subdevice=0x1f41
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
igb8@pci0:0:20:3:       class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1f41 subvendor=0x15d9 subdevice=0x1f41
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
ehci0@pci0:0:22:0:      class=0x0c0320 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f2c subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 USB Enhanced Host Controller'
    class      = serial bus
    subclass   = USB
ahci0@pci0:0:23:0:      class=0x010601 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f22 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 AHCI SATA2 Controller'
    class      = mass storage
    subclass   = SATA
ahci1@pci0:0:24:0:      class=0x010601 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f32 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 AHCI SATA3 Controller'
    class      = mass storage
    subclass   = SATA
isab0@pci0:0:31:0:      class=0x060100 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f38 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCU'
    class      = bridge
    subclass   = PCI-ISA
ichsmb0@pci0:0:31:3:    class=0x0c0500 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1f3c subvendor=0x15d9 subdevice=0x085a
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCU SMBus'
    class      = serial bus
    subclass   = SMBus
igb0@pci0:1:0:0:        class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0x1dcf subdevice=0x0308
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb1@pci0:1:0:1:        class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0x1dcf subdevice=0x0308
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
pcib3@pci0:2:0:0:       class=0x060400 rev=0x03 hdr=0x01 vendor=0x1a03 device=0x1150 subvendor=0x1a03 subdevice=0x1150
    vendor     = 'ASPEED Technology, Inc.'
    device     = 'AST1150 PCI-to-PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
vgapci0@pci0:3:0:0:     class=0x030000 rev=0x30 hdr=0x00 vendor=0x1a03 device=0x2000 subvendor=0x15d9 subdevice=0x085a
    vendor     = 'ASPEED Technology, Inc.'
    device     = 'ASPEED Graphics Family'
    class      = display
    subclass   = VGA
igb2@pci0:4:0:0:        class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1533 subvendor=0x15d9 subdevice=0x1533
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb3@pci0:5:0:0:        class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0x15d9 subdevice=0x1521
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb4@pci0:5:0:1:        class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0x15d9 subdevice=0x1521
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet

[BrandyWine]

Post #1 igb list does not align with the output from pciconf ??
mgmt is the 210 ?
Since the kernel has igb loaded and the driver is mapping to 350 and 354 and the 210, I see no reason why the system (OS) cant see them, unless something is wrong with the driver being used (see below).

pciconf shows
igb
2 210
3 350
4 350
5 354
6 354
7 354
8 354

And when I dig on it, igb is not listed in 14.3 supported list???
They show em as the driver handling i350 and i354 and i210.
Hmmmmm (scratch head).
https://www.freebsd.org/releases/14.3R/hardware/#ethernet

BUT, Intel docs show igb does support 350 354 and 210. I suspect the freebsd doc is wrong.

So, someone needs to dig more than I have.

[pfry]

Huh! Yeah, I'd expect the em driver... but then igb and em seem oddly interchangeable.

ifconfig may also show the devices. My concern was this line in their lame excuse for a data sheet: "3 x 1G bypass bridge pair", which usually indicates the presence of a hardware bypass, which must be deactivated (via custom software interface) to use the interfaces... somewhat normally. So the drivers may load, the interfaces may be configurable, but they will be electrically isolated by the bypass. But that's a supposition - I didn't find better docs offhand.

[BrandyWine]

If there is an actual bypass in that box, it's an inline L1 switch chip. I wonder what the serial console is, does the bios/uefi have the bypass util in it?

The OS will always see the logical part of the 350 354, but the actual L1 can be diverted and the OS wont even know it, and, getting actual connectivity through the nic to OS will be lost.

So maybe some more investigating the actual hardware?

[BrandyWine]

Some more on that hardware. Is it a Lanner mobo? If so what model #?
There's a SMBus controller in pciconf, this could be how bypass is controlled once OS boots.

dmesg|grep -Ei "bypass|watchdog"

The relays could be commanded from OS via gpio, this would need investigating as I suspect there's nothing in OPNsense distro supporting this bypass out-of-the-box.

2nd, online info suggests the bios should have a util for setting the default state of the bypass.


Was the bypass even tested with the OS running? Two laptops with IP in same /24, two x-over cables, laptops each connect to the matching bridge ports (looks like AAA over BBB), so any vertical pair. Can the laptops ping each other? If no x-over cables then two small switches, or one switch that can be managed to have two vlans, A-port and laptop-A on one vlanID, B-port and laptop-B on the other vlanID.

Depending on what the bios has available, it may be difficult to control the bypass on freeBSD. Can probably find some source code to compile, or even an existing klm, so more digging is needed.