Hello everyone,
Decisio DEC4040 with 2x25G SFP28 ports (ice0,1) are in a lagg (lagg0) with a switch (S1). Various VLAN subnets and derivative interfaces rules etc depend on lagg0. This lagg is the "main" LAN connection. 100G breakout DAC is used. I need to move the 2 physical connections to a different switch (S2) in the short-term and later in the medium-term split the connections across two switches (S2,3) which are in a vPC configuration. For the short-term move, I have tried the following steps neither of which have yielded a successful outcome:
- Move the 100G cable from S1 to S2. LACP isn't re-established. "No carrier" status is shown on OpnSense
- Move the 100G cable from S1 to S2 after "ifconfig lagg0 down". LACP isn't re-established. "No carrier" status is shown on OpnSense
- Move the 100G cable from S1 to S2 after bringing down all involved interfaces (lagg0,ice0,ice1).LACP isn't re-established. "No carrier" status is shown on OpnSense
- Put new 100G QSFP on S2 + 2x25G SFP28 on DEC4040 and connected with MTP-to-LC breakout cable. Repeat combination of the above steps wrt ifconfig up/down. LACP isn't re-established. "No carrier" status is shown on OpnSense
Start status of lagg0
ifconfig -v ice0
ice0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
ether f4:90:ea:00:9f:72
inet6 fe80::f690:eaff:fe00:a206%ice0 prefixlen 64 scopeid 0x5
media: Ethernet autoselect (25G-AUI <full-duplex>)
status: active
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
drivername: ice0
plugged: SFP/SFP+/SFP28 25GBASE-CR CA-25G-S (Copper pigtail)
vendor: CISCO-LEONI PN: L45593-D278-B30 SN: LCC2506GADX-CH3 DATE: 2021-02-10
root@MorikCage:~ # ifconfig -v lagg0
lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
description: main_LAGG (opt1)
options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
ether f4:90:ea:00:9f:72
hwaddr 00:00:00:00:00:00
inet 192.168.98.1 netmask 0xffffff00 broadcast 192.168.98.255
inet6 fe80::f690:eaff:fe00:9f72%lagg0 prefixlen 64 scopeid 0xd
laggproto lacp lagghash l2,l3,l4
lagg options:
flags=0<>
flowid_shift: 16
lagg statistics:
active ports: 2
flapping: 0
lag id: [(8000,F4-90-EA-00-9F-72,09A8,0000,0000),
(8000,E8-0A-B9-75-49-87,0001,0000,0000)]
laggport: ice0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,F4-90-EA-00-9F-72,09A8,8000,0005),
(8000,E8-0A-B9-75-49-87,0001,8000,01C3)]
laggport: ice1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,F4-90-EA-00-9F-72,09A8,8000,0006),
(8000,E8-0A-B9-75-49-87,0001,8000,01C4)]
groups: lagg FG_ALL_VLANs FG_CRITICAL_LAN
media: Ethernet autoselect
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
drivername: lagg0
interface port-channel9
switchport mode trunk
mtu 9216
interface Ethernet1/9/1
switchport mode trunk
mtu 9216
channel-group 9 mode active
interface Ethernet1/9/2
switchport mode trunk
mtu 9216
channel-group 9 mode active
in each of the above cases for when moving lagg0 was
ifconfig -vv lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
description: main_LAGG (opt1)
options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
ether f4:90:ea:00:9f:72
hwaddr 00:00:00:00:00:00
inet 192.168.98.1 netmask 0xffffff00 broadcast 192.168.98.255
inet6 fe80::f690:eaff:fe00:9f72%lagg0 prefixlen 64 scopeid 0xd
laggproto lacp lagghash l2,l3,l4
lagg options:
flags=0<>
flowid_shift: 16
lagg statistics:
active ports: 0
flapping: 0
lag id: [(0000,00-00-00-00-00-00,0000,0000,0000),
(0000,00-00-00-00-00-00,0000,0000,0000)]
laggport: ice0 flags=0<> state=41<ACTIVITY,DEFAULTED>
[(8000,F4-90-EA-00-9F-72,8005,8000,0005),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
laggport: ice1 flags=0<> state=41<ACTIVITY,DEFAULTED>
[(8000,F4-90-EA-00-9F-72,8006,8000,0006),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
groups: lagg FG_ALL_VLANs FG_CRITICAL_LAN
media: Ethernet autoselect
status: no carrier
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
drivername: lagg0
I think I may be missing something basic here. When changing lagg endpoints, does FreeBSD require removal and re-addition of the physical interfaces? Or perhaps some another subtlety I may be overlooking?
There should be nothing in need of change or reconfiguration on the FreeBSD side. Are you sure the ports on the new switch are configured for LACP?
Quote from: Patrick M. Hausen on December 09, 2024, 04:53:42 PM
There should be nothing in need of change or reconfiguration on the FreeBSD side. Are you sure the ports on the new switch are configured for LACP?
Thank you @Patrick. S2 has the same port-channel configuration as S1. Switch-side configuration can be found below. Initial post is updated w/ it as well.
interface port-channel9
switchport mode trunk
mtu 9216
interface Ethernet1/9/1
switchport mode trunk
mtu 9216
channel-group 9 mode active
interface Ethernet1/9/2
switchport mode trunk
mtu 9216
channel-group 9 mode active
For Cisco NX-OS series switches,
mode active engages LACP protocol negotiation from the switch.