CARP split brian issues

[raghunath.nagepalli]

Hello Experts,
I'm from Nutanix, and we are using OPNsense for one of the VPCs (internal to Nutanix, not AWS). We are observing that at certain intervals, the status of WAN and LAN CARP interfaces splits from Master in one Firewall and backup in another.

Expected behavior: Both the interfaces should always be in Master status in Primary Firewall
Ongoing behavior: Abruptly demotes the status from Master to Backup

Does anyone know what could be the reason and tips to solve to solve this issue? Is this related to version (version being used - OPNsense 22.7.4-amd64)?

[mimugmail]

Screenshots of HA settings of both FWs and logs please

[rainerle]

Are you using LAGG network devices in your setup? Just don't.

Cabeling proposal:

[Patrick M. Hausen]

@rainerle - why?

My main systems use lagg to connect to the layer 2 infrastructure and VLANs for all interfaces. No issues with CARP whatsoever. Only the HA sync interface is just a direct cable.

[rainerle]

@pmhausen
So you have one leg of the LAGG to one switch and another leg to the other switch.
Check what happens when you reboot one switch - I had only problems...
https://forum.opnsense.org/index.php?topic=14374.0

[Patrick M. Hausen]

The switches need to esplicitly support multi chassis LACP. The feature is also called "stacking".

If they don't, that would explain the problems you observe.

[rainerle]

I do expect our Huawei CE6810-32T16S4Q-LI switches to support stacking. At least the documentation says so...
https://support.huawei.com/enterprise/en/doc/EDOC1100137947/4154cda5/stacked-device-and-version-requirements

[nzkiwi68]

I would slow down your CARP messages. The stacking might be working, but, occasionally be a "bit too slow".

Set on each of the CARP virtual IPs base 2 or 3. That is slowing down the CARP to 2 or 3 seconds and see if you then get stability.

Interfaces: Virtual IPs: Settings
Advertising Frequency - Base