OPNsense Forum

English Forums => Hardware and Performance => Topic started by: zwck on November 09, 2018, 06:48:34 am

Title: Fiber Cards (intel x520DA2)
Post by: zwck on November 09, 2018, 06:48:34 am
Hello Interested People,

I just changed my hardware configuration from

ISP(Fiber) -> MediaConverter (fiber to rj45) -> OPNsense

to:

ISP(fiber) -> OPNsense (ix0) (Intel 520 DA2)

I am also online like this, however i noticed in the GUI that the negotiated speeds is set to unknown, and if config shows the same. Is that an issue

Code: [Select]
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
        ether 00:0e:c4:d2:66:77
        hwaddr 00:1b:21:9c:06:d8
        inet6 fe80::20e:c4ff:fed2:7777%ix0 prefixlen 64 scopeid 0x1
        inet6 2a02:168:2000:e:20e:c4ff:fed2:8989 prefixlen 64 autoconf
        inet 11.11.11.11 netmask 0xffffff00 broadcast 11.11.11.255
        inet 2.1.1.2 netmask 0xffffffff broadcast 2.1.1.2
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect (Unknown <rxpause,txpause>)
        status: active
(i changed my inet ip in this printout)
media: Ethernet autoselect (Unknown <rxpause,txpause>) is what i am refering to
Title: Re: Fiber Cards (intel x520DA2)
Post by: bartjsmit on November 09, 2018, 08:20:14 am
I suspect this is down to the FreeBSD driver for the fibre adapter.

If you have another fibre attached host, you can run iperf from OPNsense to test the speed locally.

Bart...
Title: Re: Fiber Cards (intel x520DA2)
Post by: mimugmail on November 09, 2018, 08:36:54 am
dmesg | grep ix
ifconfig -vvvvv ix0

I'd guess it's ok ..
Title: Re: Fiber Cards (intel x520DA2)
Post by: weust on November 09, 2018, 11:40:40 am
Isn't it possible that the media converter messes things up for FreeBSD?

Also, a Intel 520 DA2 can't do fiber, only Direct Attached Twin Axial cables.
Keep that in mind before plugging a fiber optic in it ;-)
https://ark.intel.com/products/39776/Intel-Ethernet-Converged-Network-Adapter-X520-DA2

Title: Re: Fiber Cards (intel x520DA2)
Post by: zwck on November 12, 2018, 02:41:03 pm
I mean its working fine right now. Also, i get my line speeds. I was just wondering why it sates no speeds

Intel X520-DA2 82599ES (is the modelnumber)

@mimugmail
my outputs are


Code: [Select]
acpi0: Power Button (fixed)
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e020-0xe03f mem 0xf7c80000-0xf7cfffff,0xf7d04000-0xf7d07fff irq 16 at device 0.0                                                                                                                      on pci1
ix0: Using MSI-X interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:9c:06:d8
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 4/2048, RX 4/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e000-0xe01f mem 0xf7b80000-0xf7bfffff,0xf7d00000-0xf7d03fff irq 17 at device 0.1                                                                                                                      on pci1
ix1: Using MSI-X interrupts with 5 vectors
ix1: Ethernet address: 00:1b:21:9c:06:da
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 4/2048, RX 4/2048
vlan0: changing name to 'ix1_vlan69'
vlan1: changing name to 'ix1_vlan10'
vlan2: changing name to 'ix1_vlan5'
ix1: link state changed to UP
ix1_vlan69: link state changed to UP
ix1_vlan5: link state changed to UP
ix1_vlan10: link state changed to UP
ix0: link state changed to UP
ifa_maintain_loopback_route: insertion failed for interface ix0: 17
ifa_maintain_loopback_route: insertion failed for interface ix0: 17
ix0: link state changed to DOWN
ix0: link state changed to UP
ix0: link state changed to DOWN
ix1: link state changed to DOWN
ix1_vlan69: link state changed to DOWN
ix1_vlan5: link state changed to DOWN
ix1_vlan10: link state changed to DOWN
acpi0: Power Button (fixed)
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e020-0xe03f mem 0xf7c80000-0xf7cfffff,0xf7d04000-0xf7d07fff irq 16 at device 0.0                                                                                                                      on pci1
ix0: Using MSI-X interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:9c:06:d8
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 4/2048, RX 4/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e000-0xe01f mem 0xf7b80000-0xf7bfffff,0xf7d00000-0xf7d03fff irq 17 at device 0.1                                                                                                                      on pci1
ix1: Using MSI-X interrupts with 5 vectors
ix1: Ethernet address: 00:1b:21:9c:06:da
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 4/2048, RX 4/2048
vlan0: changing name to 'ix1_vlan69'
vlan1: changing name to 'ix1_vlan10'
vlan2: changing name to 'ix1_vlan5'
ix1: link state changed to UP
ix1_vlan69: link state changed to UP
ix1_vlan5: link state changed to UP
ix1_vlan10: link state changed to UP
arp: 69.0.0.60 moved from c0:ee:fb:df:7f:70 to 0c:98:38:c4:85:59 on ix1_vlan69
arp: 192.168.0.50 moved from 02:ff:60:77:1d:58 to 02:ff:60:ea:67:9c on ix1_vlan1                                                                                                                     0
ix0: link state changed to UP
ix0: link state changed to DOWN
ix1: link state changed to DOWN
ix1_vlan69: link state changed to DOWN
ix1_vlan5: link state changed to DOWN
ix1_vlan10: link state changed to DOWN
acpi0: Power Button (fixed)
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e020-0xe03f mem 0xf7c80000-0xf7cfffff,0xf7d04000-0xf7d07fff irq 16 at device 0.0                                                                                                                      on pci1
ix0: Using MSI-X interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:9c:06:d8
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 4/2048, RX 4/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e000-0xe01f mem 0xf7b80000-0xf7bfffff,0xf7d00000-0xf7d03fff irq 17 at device 0.1                                                                                                                      on pci1
ix1: Using MSI-X interrupts with 5 vectors
ix1: Ethernet address: 00:1b:21:9c:06:da
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 4/2048, RX 4/2048
vlan0: changing name to 'ix1_vlan69'
vlan1: changing name to 'ix1_vlan10'
vlan2: changing name to 'ix1_vlan5'
ix0: link state changed to UP
ix0: link state changed to DOWN
ix0: link state changed to UP
ix1: link state changed to UP
ix1_vlan69: link state changed to UP
ix1_vlan5: link state changed to UP
ix1_vlan10: link state changed to UP
ix0: link state changed to DOWN
ix0: link state changed to UP
ix0: link state changed to DOWN
ix0: link state changed to UP
ix0: link state changed to DOWN
ix1: link state changed to DOWN
ix1_vlan69: link state changed to DOWN
ix1_vlan5: link state changed to DOWN
ix1_vlan10: link state changed to DOWN
acpi0: Power Button (fixed)
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e020-0xe03f mem 0xf7c80000-0xf7cfffff,0xf7d04000-0xf7d07fff irq 16 at device 0.0                                                                                                                      on pci1
ix0: Using MSI-X interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:9c:06:d8
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 4/2048, RX 4/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x                                                                                                                     e000-0xe01f mem 0xf7b80000-0xf7bfffff,0xf7d00000-0xf7d03fff irq 17 at device 0.1                                                                                                                      on pci1
ix1: Using MSI-X interrupts with 5 vectors
ix1: Ethernet address: 00:1b:21:9c:06:da
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 4/2048, RX 4/2048
vlan0: changing name to 'ix1_vlan69'
vlan1: changing name to 'ix1_vlan10'
vlan2: changing name to 'ix1_vlan5'
ix0: link state changed to UP
ix1: link state changed to UP
ix1_vlan69: link state changed to UP
ix1_vlan5: link state changed to UP
ix1_vlan10: link state changed to UP


and

Code: [Select]
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
        ether 00:0e:c4:d2:89:89
        hwaddr 00:1b:21:9c:06:d8
        inet6 fe80::%ix0 prefixlen 64 scopeid 0x1
        inet6 2a02: prefixlen 64 autoconf
        inet ip.ip.ip.ip netmask 0xffffff00 broadcast 212.51.143.255
        inet 2.1.1.2 netmask 0xffffffff broadcast 2.1.1.2
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect (Unknown <rxpause,txpause>)
        status: active
        plugged: SFP/SFP+/SFP28 1000BASE-LX (LC)
        vendor: OEM PN: SFP-BX-U31-IN SN: BOGU239 DATE: 2018-01-23
        module temperature: 44.84 C Voltage: 3.30 Volts
        RX: 0.05 mW (-12.51 dBm) TX: 0.23 mW (-6.23 dBm)

        SFF8472 DUMP (0xA0 0..127 range):
        03 04 07 00 00 00 02 12 00 01 01 01 0D 00 14 C8
        00 00 00 00 4F 45 4D 20 20 20 20 20 20 20 20 20
        20 20 20 20 00 00 00 00 53 46 50 2D 42 58 2D 55
        33 31 2D 49 4E 20 20 20 31 2E 30 20 05 1E 00 1B
        00 1A 00 00 42 4F 47 55 32 33 39 20 20 20 20 20
        20 20 20 20 31 38 30 31 32 33 20 20 68 90 01 6D
        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

for my rj45 sfp module

Code: [Select]
ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c500b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,TXCSUM_IPV6>
        ether 00:1b:21:9c:06:da
        hwaddr 00:1b:21:9c:06:da
        inet6 fe80::21b:21ff:fe9c:6da%ix1 prefixlen 64 scopeid 0x2
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        plugged: SFP/SFP+/SFP28 1000BASE-T (RJ45)
        vendor: OEM PN: GLC-T SN: C1809270106 DATE: 2018-09-30

        SFF8472 DUMP (0xA0 0..127 range):
        03 04 22 00 00 00 08 00 00 00 00 01 0D 00 00 00
        00 00 64 00 4F 45 4D 20 20 20 20 20 20 20 20 20
        20 20 20 20 00 00 00 00 47 4C 43 2D 54 20 20 20
        20 20 20 20 20 20 20 20 41 20 20 20 00 00 00 7C
        00 1A 00 00 43 31 38 30 39 32 37 30 31 30 36 20
        20 20 20 20 31 38 30 39 33 30 20 20 00 00 00 74
        00 00 11 55 AD 3D D1 1F E3 F6 0B 6B 97 FF 8D 46
        80 75 CD 00 00 00 00 00 00 00 00 00 A2 CB 7B E2

Title: Re: Fiber Cards (intel x520DA2)
Post by: mimugmail on November 12, 2018, 02:43:28 pm
It just doesn't recognize the correct firmware of the Gbic ...
Title: Re: Fiber Cards (intel x520DA2)
Post by: zwck on November 12, 2018, 02:55:57 pm
anything i can do about this?

At the moment i have two different sfp modules in there that are compatible.
one   Lodfiber ABCU 5710RZ 1000Base-T Sfp Copper RJ-45 100 m Transceiver
one    Intel SFP-BIDI-1310  BlueOptics SFP BO15C3155620D
Title: Re: Fiber Cards (intel x520DA2)
Post by: zwck on November 13, 2018, 08:55:47 am
Isn't it possible that the media converter messes things up for FreeBSD?

Also, a Intel 520 DA2 can't do fiber, only Direct Attached Twin Axial cables.
Keep that in mind before plugging a fiber optic in it ;-)
https://ark.intel.com/products/39776/Intel-Ethernet-Converged-Network-Adapter-X520-DA2

Its weird that you mention this and yet, both, the Ethernet SFP and the BIDI device, work without a flaw.
Title: Re: Fiber Cards (intel x520DA2)
Post by: mimugmail on November 13, 2018, 10:10:29 am
DA stands for Direct Attach, so you can easily use a TwinAx cable where the Gibc is fixed on the cable.
There are also models with a Gbic integrated where such a cable wouldn't fit.

But all this doesn't mean that you cant use a normal Gbic with this card ...
Title: Re: Fiber Cards (intel x520DA2)
Post by: zwck on November 13, 2018, 03:17:54 pm
I am still confused, probably because i am new to >gbit networks.

I don't use the card with a direct attached twin cables. I use supported sfp modules from https://www.intel.com/content/www/us/en/support/articles/000005528/network-and-i-o/ethernet-products.html
to have normal ethernet and fiber capabilities. One SFP slot has a 1.25gbit fiber module plugged in.
Title: Re: Fiber Cards (intel x520DA2)
Post by: weust on November 13, 2018, 07:46:11 pm
This is the first time I've read/seen that DA cards can use optics.
But if it's original Intel and it works, that is very cool.