I set up CARP using the OPNsense docs, and it mostly works; the firewalls sync and failover correctly. For the sake of example, let's say my setup has the same WAN IPs as the OPNsense docs:
Primary | 172.18.0.101/24 |
Secondary | 172.18.0.102/24 |
Virtual IP | 172.18.0.100/24 |
I've made a manual outbound NAT rule with the following settings:
Interface | WAN |
Source | any |
Source Port | * |
Destination | * |
Destination Port | * |
NAT Address | 172.18.0.100 |
NAT Port | * |
Static Port | NO |
However there are two major problems:
- When the primary firewall comes back up, the secondary firewall will not relinquish master status. The secondary-master must be brought down/rebooted for the primary to reclaim CARP master.
- Regardless of which firewall is currently the backup, its WAN interfaces are perpetually down. This seems to be because it is trying to use the WAN virtual IP, but that IP is already used by the current master.
I tried everything I could think of to fix this, and eventually I found the following note in the pfSense CARP docs:
QuoteNever add outbound NAT rules that could match the WAN/Public IP addresses of the cluster. This includes both rules that have the public IP addresses listed explicitly and also rules that have any set as a source. These NAT rules will cause other problems/unintended behavior, and will break outbound connectivity from the secondary node when it is in a BACKUP state.
This exactly describes at least one of my problems. Assuming "the WAN/Public IP addresses of the cluster" would refer to 172.18.0.100, this seems to be at odds with the OPNsense CARP docs, which state the following:
QuoteGo to Firewall -> NAT and select outbound nat. Choose manual outbound nat on this page and change the rules originating from the 192.168.1.0/24 network to use the CARP virtual interface (172.18.0.100).
So, if you aren't supposed to use the WAN virtual IP, which NAT address should be used to set up outbound NAT correctly?