Duplicated data flow

Started by GreenMatter, March 23, 2026, 08:42:34 PM

Previous topic - Next topic
With your assistance in previous topics, I got HA in working condition, but...

To describe my setup:
  • 2x Opnsense instances in high availability mode with carp vip interfaces on single pve host. I know it's not full HA but I want software HA and also simply to test it.
  • VMs are connected through 3 bridges: 1 on WAN side, the other on LAN side (and further trunk physical link to switch) and pfsync bridge.
  • IGMP snooping, storm control are disabled in (UniFi) switches.

In order to change above configuration and (trying to) test my issue, I created additional LAN bridge for backup instance and instead of having them (2x opnsense) connected over single linux bridge - within proxmox, I connected them over physical switch.
This of course requires second downlink:
  • master/regular LAN bridge would remain connected as it is now
  • backup/new LAN bridge is connected to switch via additional downlink

But problem I'm facing is duplicated communication/data flow to and from both VMs; both instances have same looking graphs in proxmox webgui - network flow and also cpu. Despite they don't change their master/backup status (no flapping at carp status) I have something similar to split brain situation, for example if I communicate with opnsense webgui or ssh on carp vip interface, reply comes either from one of those two and toggles every few seconds. If I ping them, reply is duplicated ("DUP!"). Communication to other hosts and WAN is ok.  I have already set Mac filter to "no" in proxmox VM's firewall options (pve firewall is disabled). I tried ovs and Linux bridges with same results.

To me, it is something related to MAC and network switches; is it possible to set it up correctly?

OPNsense on:
Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (4 cores)
8 GB RAM
50 GB HDD
and plenty of vlans ;-)

One more thing to add is when I reach my LAN over VPN (either Wireguard or OpenVPN) I can't communicate with backup instance (its physical interface addresses) at all while FW rules allow them to send requests to any hosts...
OPNsense on:
Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (4 cores)
8 GB RAM
50 GB HDD
and plenty of vlans ;-)

So, for those who may experience similar issues:
  • WG (only in HA CARP mode) requires additional outbound rule for LAN interfaces and source as WG net
  • Very unexpected reason of duplicated data was my set up of CARP VIPs in unicast mode. After removing peers (back to multicast CARP), all of sudden there's no more duplicated data flow and pings plus no webgui/ssh flapping... I wonder why?
OPNsense on:
Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (4 cores)
8 GB RAM
50 GB HDD
and plenty of vlans ;-)