Public IP-Range as VIP/CARP

[Hunduster]

Hello all,

we just replaced two Sophos XG with two OPNsense in the data center.

In the data center there are only the two firewalls, a switch and a NAS which takes our backup. For this reason there is also a small /29 subnet here.

Currently the config looks like this:

OPNsense VIP: 172.16.21.1/29
Switch: 172.16.21.2/29
OPNsense node1: 172.16.21.3/29
OPNsense node2: 172.16.21.4/29
NAS: 172.16.21.5/29

Everything works fine so far.

Now we have received a public IP range /29 from the data center.

Currently the WAN interfaces are configured as follows:

OPNsense node1: 88.74.150.2
OPNsense node1: 88.74.150.3

the other Public-IP addresses are stored as CARP VIP.

My question is, if I can configure the first two IPs (88.74.150.2, 88.74.150.3) also as CARP.

My thought behind this is that if the cluster swapped, node2 is also reachable with 88.74.150.2. However, I am currently not sure what I need to configure on the physical interface if I want to use ALL public IPs as CARP.

[Monviech (Cedrik)]

That won't work.

The physical interfaces always need the non virtual IPv4 address. That means in your /29 subnet, two IP addresses are unusable for CARP because they're used by the physical interfaces of the cluster.

If a node fails, you will be able to reach the other node with one of the existing CARP VIPs, because they will automatically migrate to the other firewall.

[franco]

> That won't work.

Truth be told try setting IPv4 WAN mode to "none" and add a CARP virtual IP to WAN on both machines and see what happens.


Cheers,
Franco

[Monviech (Cedrik)]

> That won't work.

Truth be told try setting IPv4 WAN mode to "none" and add a CARP virtual IP to WAN on both machines and see what happens.


Cheers,
Franco

I guess that works because it's described in the pfsense documentation:
https://docs.netgate.com/pfsense/en/latest/solutions/reference/highavailability/prerequisites.html#single-address-carp

But they don't recommend it because one firewall won't be connected to the WAN.

[franco]

It is connected but you can't reach it directly from WAN, but SSHing into the master or into a device in LAN will help you to access the other one.

It really depends on the constraints given by the ISP... some only allow /32 so what do you want to do instead if HA is a requirement.


Cheers,
Franco

[Monviech (Cedrik)]

If the layer 2 network is ever having some issues with the carp broadcasts, the CARP VIPs could flap between the firewalls. And I would imagine that the WEB Gui and SSH from WAN to the opnsense wouldn't work right anymore if there is a constant master/backup switch with the CARP VIPs flapping between them.

What I don't know is if there's a mechanism that prevents flapping, e.g. by demoting one node to the point it can't become master anymore.

I would only recommend it if there is Out Of Band management.

That's just my opinion of course.

[franco]

It could happen, but in practice this is rarely the case and you have other issues anyway.


Cheers,
Franco