Messages

1

Development and Code Review / Re: Support for Intel X553 NIC and Marvell 88E6190 switch

« on: March 27, 2023, 09:45:05 pm »

Hey Patrick,

I've tried compiling my own kernel with various options enabled. Sadly I'm pretty sure it's not going to work "out of the box" without further changes.

As I mentioned in the original post looking at the code I see the following issues:

1. The e6000sw device uses FDT (Flattened Device Tree) which, unless I'm mistaken, is not available for amd64. I think I can work around this with some "hacks" using hints instead (similar hints as those mentioned in the pfSense post).
2. The ixgbe device doesn't seem to be setting up the MDIO connection to the switch.

Number 2 is the first thing I need to solve.

I found this interesting page: https://wiki.freebsd.org/StefanBethke/EtherSwitch

I've been looking at the code of sys/mips/atheros/if_arge.c and can see lots of MDIO stuff which doesn't seem to have an equivalent in the ixgbe driver.

Regards,
Ivan

2

Development and Code Review / Re: Support for Intel X553 NIC and Marvell 88E6190 switch

« on: March 27, 2023, 05:18:39 pm »

Just as an FYI I set up pfSense 23.01 on my appliance. I've captured the relevant part of the boot log to show how it detects the switch. I setup the hints as described in the link my previous post.

ix0: <Intel(R) X553 (KR Backplane)> mem 0xdfa00000-0xdfbfffff,0xdfc04000-0xdfc07fff irq 16 at device 0.0 on pci6
ix0: Using 2048 TX descriptors and 2048 RX descriptors
ix0: Using 4 RX queues 4 TX queues
ix0: Using MSI-X interrupts with 5 vectors
ix0: allocated for 4 queues
ix0: allocated for 4 rx queues
ix0: Ethernet address: 00:a0:c9:00:00:02
ix0: eTrack 0x80000863
ix0: netmap queues/slots: TX 4/2048, RX 4/2048
ix1: <Intel(R) X553 (KR Backplane)> mem 0xdf800000-0xdf9fffff,0xdfc00000-0xdfc03fff irq 17 at device 0.1 on pci6
ix1: Using 2048 TX descriptors and 2048 RX descriptors
ix1: Using 4 RX queues 4 TX queues
ix1: Using MSI-X interrupts with 5 vectors
ix1: allocated for 4 queues
ix1: allocated for 4 rx queues
ix1: Ethernet address: 34:12:78:56:01:03
ix1: eTrack 0x80000863
mdio0: <MDIO> on ix1
e6000sw0: <Marvell 88E6190> on mdio0
e6000sw0: single-chip addressing mode
miibus0: <MII bus> on e6000sw0
e1000phy0: <Marvell 88E1000 Gigabit PHY> PHY 1 on miibus0
e1000phy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus1: <MII bus> on e6000sw0
e1000phy1: <Marvell 88E1000 Gigabit PHY> PHY 2 on miibus1
e1000phy1:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus2: <MII bus> on e6000sw0
e1000phy2: <Marvell 88E1000 Gigabit PHY> PHY 3 on miibus2
e1000phy2:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus3: <MII bus> on e6000sw0
e1000phy3: <Marvell 88E1000 Gigabit PHY> PHY 4 on miibus3
e1000phy3:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus4: <MII bus> on e6000sw0
e1000phy4: <Marvell 88E1000 Gigabit PHY> PHY 5 on miibus4
e1000phy4:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus5: <MII bus> on e6000sw0
e1000phy5: <Marvell 88E1000 Gigabit PHY> PHY 6 on miibus5
e1000phy5:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus6: <MII bus> on e6000sw0
e1000phy6: <Marvell 88E1000 Gigabit PHY> PHY 7 on miibus6
e1000phy6:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus7: <MII bus> on e6000sw0
e1000phy7: <Marvell 88E1000 Gigabit PHY> PHY 8 on miibus7
e1000phy7:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
e6000sw0: switch is ready.
etherswitch0: <Switch controller> on e6000sw0
ix1: netmap queues/slots: TX 4/2048, RX 4/2048

3

Development and Code Review / Support for Intel X553 NIC and Marvell 88E6190 switch

« on: March 13, 2023, 12:57:32 pm »

I have recently acquired a WatchGuard Firebox M270 and would like to run opnsense on it.

The motherboard is a Lanner NCB-WG2511A v0.5.
The processor is Intel(R) Atom(TM) CPU C3558
This includes the Intel X553 integrated dual-port 10 GbE LAN controller (see here: https://www.intel.com/content/www/us/en/products/docs/processors/atom/c-series/atom-c3000-family-programmer-reference-manual.html)
It also appears to have a Marvell 88E6190 switch (see here: https://studylib.net/doc/25668514/88e6390)
This has the following interfaces:
x8: 10/100/1000BASE-T
x2: SERDES (2.5G/1G)
x1: MII/RMII/RGMII

opnsense is detecting the X553

ix0: <Intel(R) X553 (KR Backplane)> mem 0xdfa00000-0xdfbfffff,0xdfc04000-0xdfc07fff irq 16 at device 0.0 on pci6
ix0: Using 2048 TX descriptors and 2048 RX descriptors
ix0: Using 4 RX queues 4 TX queues
ix0: Using MSI-X interrupts with 5 vectors
ix0: allocated for 4 queues
ix0: allocated for 4 rx queues
ix0: Ethernet address: 00:a0:c9:00:00:02
ix0: eTrack 0x80000863
ix0: netmap queues/slots: TX 4/2048, RX 4/2048
ix1: <Intel(R) X553 (KR Backplane)> mem 0xdf800000-0xdf9fffff,0xdfc00000-0xdfc03fff irq 17 at device 0.1 on pci6
ix1: Using 2048 TX descriptors and 2048 RX descriptors
ix1: Using 4 RX queues 4 TX queues
ix1: Using MSI-X interrupts with 5 vectors
ix1: allocated for 4 queues
ix1: allocated for 4 rx queues
ix1: Ethernet address: 34:12:78:56:01:03
ix1: eTrack 0x80000863
ix1: netmap queues/slots: TX 4/2048, RX 4/2048

It is not however detecting the switch. In an ideal world I'd expect to see /dev/etherswitch0 appearing. I don't see any MDIO messages suggesting that this isn't being set up correctly.

The WatchGuard runs software based on Linux (I can share a boot log if it helps to see the devices it detects).
More interestingly it appears that pfSense can be got to run on the device. See here: https://forum.netgate.com/topic/154974/pfsense-on-watchguard-m270

This seems to suggest they have made some changes to the drivers to handle this hardware. i don't however believe those changes are publicly available anywhere, which is a shame. I don't see anything in FreeBSD for example reflecting the hints mentioned in the pfSense post.

Looking at the code I see the following issues:

1. The e6000sw device uses FDT which is not available for amd64. I think I can work around this with some "hacks" using hints instead (similar hints as those mentioned in the pfSense post).
2. The ixgbe device doesn't seem to be setting up the MDIO connection to the switch.

I don't really know what way this should work but as an experiment I added the following code in  /usr/src/sys/dev/ixgbe/if_ix.c at the end of the ixgbe_if_attach_post function. My idea was to use a similar hint as mentioned in the pfSense post hint.mdio.0.at="ix1" to tell the ixgbe driver that it needs to setup the mdio connection to allow the switch to be found.


        if ( 0 == resource_string_value( "mdio", 0, "at", &mdio_dev ) ) {
                sprintf(dev_name, "%s%d", device_get_name(dev), device_get_unit(dev));
                if( 0 == strcmp( mdio_dev, dev_name ) ) {
                        device_printf(dev, "hint found for mdio on %s - adding child device\n", dev_name);
                        child = device_add_child( dev, "mdio", 0 );
                        if (child == NULL) {
                                device_printf(dev, "creating mdio child device failed\n");
                        } else {
                                retval = bus_generic_attach(dev);
                                device_printf(dev, "bus_generic_attach(dev) returned %d\n", retval);
                                retval = bus_generic_attach(child);
                                device_printf(dev, "bus_generic_attach(child) returned %d\n", retval);
                        }

                }

This appears to execute successfully. devinfo -rv is showing mdio0 but it is not attached. It is still not detecting the switch. I put some debug in /usr/src/sys/dev/mdio/mdio.c and it doesn't appear that any of the functions are running e.g. probe, attach, identify.

I'd appreciate any ideas on what needs to be done to get this working!