one of the IP refused to be backup

[Edy]

Type   opnsense-business   
Version   23.10.2   
Architecture   amd64   
Commit   212ca5a53

2x units HA purchased from Deciso DEC2752EU

Configured Carp and for some reason one of the CARP IP refused to be Backup and now I have 2 Master CARP IP which is a problem because it is being used as a DHCP Gateway IP.

It was working before and after adding acme and did a HA sync this become a problem

Where do i begin?

[Monviech (Cedrik)]

Make a backup of both firewalls and look at the <virtualip version="1.0.0"> section in the config.xml.

Post them here (omit the password)

Like this:

Firewall Master:

Code Select Expand


  <virtualip version="1.0.0">
    <vip uuid="211c9748-651a-40cf-bbd6-918cedd845ec">
      <interface>opt1</interface>
      <mode>carp</mode>
      <subnet>172.16.0.254</subnet>
      <subnet_bits>24</subnet_bits>
      <gateway/>
      <noexpand>0</noexpand>
      <nobind>0</nobind>
      <password>XXXXX</password>
      <vhid>1</vhid>
      <advbase>1</advbase>
      <advskew>0</advskew>
      <descr>VIP LAN IPv4</descr>
    </vip>


Firewall Backup:

Code Select Expand


  <virtualip version="1.0.0">
    <vip uuid="211c9748-651a-40cf-bbd6-918cedd845ec">
      <interface>opt1</interface>
      <mode>carp</mode>
      <subnet>172.16.0.254</subnet>
      <subnet_bits>24</subnet_bits>
      <gateway/>
      <noexpand>0</noexpand>
      <nobind>0</nobind>
      <password>XXXXX</password>
      <vhid>1</vhid>
      <advbase>1</advbase>
      <advskew>101</advskew>
      <descr>VIP LAN IPv4</descr>
    </vip>


[Edy]

Firewall Master

<vip uuid="f43370f0-6623-44b8-b745-b14587c204bd">
      <interface>opt3</interface>
      <mode>carp</mode>
      <subnet>10.88.1.1</subnet>
      <subnet_bits>24</subnet_bits>
      <gateway/>
      <noexpand>0</noexpand>
      <nobind>0</nobind>
      <password>password</password>
      <vhid>2</vhid>
      <advbase>1</advbase>
      <advskew>0</advskew>
      <descr/>
    </vip>

Firewall Backup

<vip uuid="f43370f0-6623-44b8-b745-b14587c204bd">
      <interface>opt3</interface>
      <mode>carp</mode>
      <subnet>10.88.1.1</subnet>
      <subnet_bits>24</subnet_bits>
      <gateway/>
      <noexpand>0</noexpand>
      <nobind>0</nobind>
      <password>password</password>
      <vhid>2</vhid>
      <advbase>1</advbase>
      <advskew>100</advskew>
      <descr/>
    </vip>

[Monviech (Cedrik)]

Good that looks fine.

How are the OPNsense Firewalls connected to the Switch?

On each interface or VLAN that has a CARP VIP, there is a broadcast that happens every advbase in second + advskew in milliseconds.

If this broadcast doesn't work between the firewalls, both will think that they are master on this interface or vlan.

[Edy]

10G interface with tagged clan. Each tagged clan (LAN) has an IP and also CARP Virtual IP.

This broadcast is working because it is on the same switch

[Monviech (Cedrik)]

Are you using only tagged VLANs? Even tagged vlan1? If not, you should make sure you don't use untagged vlans (native VLAN) on trunk ports.

Does your switch do stuff like igmp snooping? If yes, I would disable that on the ports the opnsenses are connected to.

Can you see these broadcasts on both firewalls? Check on the interface with

Code Select Expand

tcpdump -i vlan0.1 proto 112 (change vlan0.1 to your interface name, proto 112 is vrrp which carp uses for broadcasts)

Are the interface assigments on both firewalls exactly the same (opt numbers matching between them for all interfaces in interface assignments)?

[Edy]

I have found out the issue. One of my co worker removed the tagged vlan hence this is not working. I am going to get him put them back tomorrow and we should be good :)

Btw is that the right practise to have a virtual carp ip per tagged vlan interface?

[Patrick M. Hausen]

Btw is that the right practise to have a virtual carp ip per tagged vlan interface?

If you want HA/failover on that VLAN interface, CARP is the way.

[Monviech (Cedrik)]

Nice job, sometimes its the little things. Now you got experience what to check too the next time it happens. ^^