No ARP / CARP IP seen by broadband routeur

[NEOSA]

Hi All@community

We are facing an issue with a CARP/HA cluster.

Environment works perfectly with the HA Cluster, last OpnSense release up2date :

- 2 identical OpenSense boxes,
- A CARP LAN woking correctly : toogle from MASTER to SLAVE while removing one member,
- Two gateways : first ISP is a fiber connection (Zyxel VMG3625-T50B routeur), second on a GSM 5G cellular WAN
- Both gateways are in a CARP WAN OpnSense setup : WAN_CARP ans 5G_CARP
- Gateways Group setup : also working in good state, setup from WAN to 5G ou 5G to WAN if any gateways are removed

Issue we met : the WAN IP fiber router sees only an ARP table with the two IP of each OpnSense boxes (192.168.1.3/24 and 192.168.1.4/24), the CARP address (192.168.1.254/24) + the CARP MacAddress is not seen :-(

We thought of an ISP router misconfiguration but no L2/L3 setup available at this side.

We added an intermediate Ethernet switch beetween the 2 OpnSense boxes and the fiber ISP routeur : same issue.

We absolutely do not have any outbound issues from the LAN side toward the Internet, but due to this fact as no ARP entries are shown at the ISP router  side, we are not able to perform any incoming traffic.

The ISP broadband routeur (Zyxel VMG3625-T50B routeur) is setup to perform NAT (not as bridge mode) : while adding a NAT rule for accepting OpenVPN traffic ==> IP address for one of the 2 OpnSense box instead of the WAN CARP IP (192.168.1.254), we ar able to open a VPN tunnel from outside. If we modify the VPN behaviour to use the WAN_CARP address + modifying the NAT entry at the Zyxel VMG3625-T50B routeur, no more any OpenVPN incoming traffic works.

As a reminder, the broadband router always shows ARP/IP address for the two OpnSense boxes.

Any clues/ideas will be welcome, we will have a look with an expert this Wednesday October 25th do go deeper, if we missed a setup.

Ciao - Au revoir

Sorry for any English mistakes, I'm not english native spoken !!!

[Monviech]

Yeah that sounds weird. The CARP VIP should be in the ARP tables with it's virtual MAC address.

For example this is one of my CARP VIPs as seen by a client:

Code: [Select]

admin@pc01:~$ arp -a 172.16.0.254
_gateway (172.16.0.254) on 00:00:5e:00:01:01 [ether] on wlp4s0

admin@pc01:~$ ip -6 neighbor
fe80::172:16:0:254 dev wlp4s0 lladdr 00:00:5e:00:01:02 router REACHABLE

Here you can see that each CARP VIP should have a unique virtual MAC address thats seen by other devices, with NDP and ARP.

[NEOSA]

Back for news ;-)

The HA Cluster has been tested in a separate way, behind an ISP router different from the first post (Sagem Livebox V5).

I added a small 8 ports switch between the ISP routeur & the WAN interface of each Opnsense box.

One modification has been done : the outbound rules where set as "Source : LAN Address" instead of "LAN Net" ==> My expert told me it wasn't relevant enough for the previous CARP_VIP/MacAddress issue we met.



For now, the HA Cluster is fully responsive, the failover works great, incoming VPN or incoming NAT rules are processed ;-) The only "trouble" I have is OpenVPN connection not kept during a failover, but this will be not a real problem for end users.

I will give you some feedback when I well go back on site.