With your assistance in previous topics, I got HA in working condition, but...
To describe my setup:
- 2x Opnsense instances in high availability mode with carp vip interfaces on single pve host. I know it's not full HA but I want software HA and also simply to test it.
- VMs are connected through 3 bridges: 1 on WAN side, the other on LAN side (and further trunk physical link to switch) and pfsync bridge.
- IGMP snooping, storm control are disabled in (UniFi) switches.
In order to change above configuration and (trying to) test my issue, I created additional LAN bridge for backup instance and instead of having them (2x opnsense) connected over single linux bridge - within proxmox, I connected them over physical switch.
This of course requires second downlink:
- master/regular LAN bridge would remain connected as it is now
- backup/new LAN bridge is connected to switch via additional downlink
But problem I'm facing is duplicated communication/data flow to and from both VMs; both instances have same looking graphs in proxmox webgui - network flow and also cpu. Despite they don't change their master/backup status (no flapping at carp status) I have something similar to split brain situation, for example if I communicate with opnsense webgui or ssh on carp vip interface, reply comes either from one of those two and toggles every few seconds. If I ping them, reply is duplicated ("DUP!"). Communication to other hosts and WAN is ok. I have already set Mac filter to "no" in proxmox VM's firewall options (pve firewall is disabled). I tried ovs and Linux bridges with same results.
To me, it is something related to MAC and network switches; is it possible to set it up correctly?