CARP breaks haproxy health checks?

[doug.dimick]

I'm running two OPNsense 18.1.3 systems with LAN/WAN/DMZ interfaces and CARP VIP.

LAN CARP VIP 192.168.1.1/17
LAN OPNsense-1 192.168.1.2/17
LAN OPNsense-2 192.168.1.3/17

DMZ CARP VIP 192.168.254.1/24
DMZ OPNsense-1 192.168.254.2/24
DMZ OPNsense-2 192.168.254.3/24

I'm running haproxy only on my master OPNsense-1 system. When OPNsense-1 is running by itself, everything works great.

When I boot OPNsense-2, everything still works great except all my haproxy HTTP health checks running on OPNsense-1 fail.

If I shut down OPNsense-2, the haproxy HTTP health checks on OPNsense-1 immediately start working again.

All traffic through either OPNsense system works fine in both scenarios. The only thing that stops working are the haproxy health checks.

What could be causing this behavior?

[doug.dimick]

I still have no solution for this. I've rebuilt my backup opnsense from scratch and it still behaves exactly the same.

Is anyone running both carp and haproxy successfully?

[doug.dimick]

I've narrowed this down to having Netflow enabled and capturing local on both master/backup. As soon as you enable it on the backup, the HAproxy health checks on the master immediately fail. Disable Netflow on the backup, and they immediately start working again.

Steps to reproduce:

1. Run two instances of OPNsense.
2. Configure high availability (I'm using CARP and XMLRPC sync, but I am not using states sync).
3. Configure HAproxy on the primary. Observe that health checks show your backends/frontends as UP.
4. Enable Netflow on master.
5. Observe that HAproxy health checks still work.
6. Enable Netflow on backup.
7. Observe that HAproxy health checks now show your backends/frontends as DOWN.