LAGG Lan interface not working after upgrade to 24.7 RC2

[jfenech]

A quick heads up, took the plunge today, and upgraded to 24.7 RC2. Upgrade went smoothly, howver after the final reboot my LAGG Lan. interface failed to come up. It showed the correct ip in console, but I could not ping it or route any traffic through the firewall.

I went back to 24.1, removed the LAGG interface and converted to a normal lan interface and upgrade went smoothly, everything is working as it should, openvpn, wg, and kea dhcp.

Just a heads up in case anyone has a LAGG lan interface it would be a good idea to switch it off before upgrading the beta.

Have not tried to re-enable lagg since, but will probably give it a try tomorrow if I find some time.


The new UI looks great. Super well done !

P.S. My Hardware is a ROUAFWIT Micro Firewall N100 12TH Gen Fanless Mini PC, DDR5 8GB RAM 128GB SSD Router Appliance Computer

[jfenech]

Did some more digging this morning, so it seems that whenever two interfaces are selected for LAGG, their mac address becomes 00:00...:00 ?

lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
   description: LAN (lan)
   options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
   ether 00:00:00:00:00:00
   inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255
   laggproto lacp lagghash l2,l3,l4
   laggport: igc1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
   laggport: igc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
   groups: lagg
   media: Ethernet autoselect
   status: active
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


-----

igc1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
   options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
   ether 00:00:00:00:00:00
   hwaddr a8:b8:e0:02:ea:cf
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igc2: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
   options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
   ether 00:00:00:00:00:00
   hwaddr a8:b8:e0:02:ea:d0
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active

Not sure if this is normal, or a bug.

[franco]

Thans for the report. Can you try with https://github.com/opnsense/core/commit/afe9c107665b ?

# opnsense-patch afe9c107665b


Cheers,
Franco

[jfenech]

I confirm it is  working !! Thank you for the super prompt reply

[franco]

It's not working? It needs a reboot I should have said.


Cheers,
Franco

[Seimus]

I can confirm that patch helped.

After applying it and rebooting LAGG shows proper inherited MAC

10.99.99.1   00:0c:29:ba:92:c9   VMware, Inc.   lagg0   LAGG_TEST

Regards,
S.

[franco]

Thanks for testing. That empty macaddr_hw makes me cringe a bit so in the future that is probably what ends up being ignored by the core: https://github.com/opnsense/core/commit/76695679

But we will give that more testing later. For 24.7 the change was a clean revert to the 24.1 behaviour for safety reasons (the mentioned and tested patch).


Cheers,
Franco

[jfenech]

Apologies... IT IS WORKING, and did not require a reboot on my end, just needed to bring the interfaces down and back up

As you said LAGG0 still has blank MAC

lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
   description: LAN (lan)
   options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
   ether a8:b8:e0:02:ea:cf
   hwaddr 00:00:00:00:00:00
   inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255
   laggproto lacp lagghash l2,l3,l4
   laggport: igc1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
   laggport: igc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
   groups: lagg
   media: Ethernet autoselect
   status: active
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


but igc1 and igc2 have proper mac addresses now even in the UI

[Seimus]

I feel the same "cringe" now when you pointed it.

Anyway many thanks for quickly providing a fix, LAGGs are in my Infra butter and bread for resilience.

Regards,
S.

[Patrick M. Hausen]

Isn't the MAC address what is in the "ether" line - which could theoretically be changed but rarely is for physical interfaces, while "hwaddr" is the fixed address in the device's ROM? Since a lagg doesn't have a fixed hardware address I'd say that makes sense.

If you look at the arp cache on any other device connected to that lagg you will surely find a8:b8:e0:02:ea:cf as the MAC address of your OPNsense.

[franco]

In order to be consistent LAGG should advertise it's "native" address under "ether" always and only show "hwaddr" when the user manually overwrote "ether" which is how everything else works. The mechanism is a bit too transparent and/or seems to use what the user would do rather than providing a transient "hwaddr" depending on which member is attached first.


Cheers,
Franco

[Wirehead]

In order to be consistent LAGG should advertise it's "native" address under "ether" always and only show "hwaddr" when the user manually overwrote "ether" which is how everything else works. The mechanism is a bit too transparent and/or seems to use what the user would do rather than providing a transient "hwaddr" depending on which member is attached first.


Cheers,
Franco

Hi Franco, sorry to disturb in busy times with the 24.7 release and related upgrade path - but is this fix included in the 24.7 final release? Asking since my main functionality depends on LAGG 

[franco]

Yes.


Cheers,
Franco

[Wirehead]

Awesome - thank you!