OPNsense Forum

English Forums => Hardware and Performance => Topic started by: Ryushin on April 13, 2025, 07:23:31 PM

Title: Mellanox ConnectX-4 CARP Multicast Not Working
Post by: Ryushin on April 13, 2025, 07:23:31 PM
We have two GoWin GW-BS-1UR2-25G:
https://www.gowinfanless.com/products/network-device/1u-2u-server/gw-bs-1ur2-25g

It seems there is a bug in the ConnectX-4 Lx driver for FreeBSD.  Broadcasts are looped back to themselves so CARP won't work in multicast.

Symptoms: Both devices will stay in CARP Backup when using multicast and neither will become Master.  Switching to unicast allows CARP to work.

Constant Duplicate Address Detected (DAD) message in system log:
2025-04-13T09:37:45-07:00 <hostname goes here) kernel - - [meta sequenceId="1585"] <3>mce0: a looped back NS message is detected during DAD for fe80:a::526b:4bff:fef0:8058.  Another DAD probes are being sent.
<13>1 2025-04-13T09:37:47-07:00 <hostname goes here) kernel - - [meta sequenceId="1586"] <3>mce1: a looped back NS message is detected during DAD for fe80:b::526b:4bff:fef0:8059.  Another DAD probes are being sent.

Pinging the CARP IP results in a DUP! message.

Links I found with problems:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270859 (Disabling SRV-IO did not work for me)
https://network.nvidia.com/pdf/prod_software/Mellanox_FreeBSD_Release_Notes_3.0.0.pdf

Anyone know of any workarounds for the broadcast problem with the Mellanox ConnectX-4?