Connectivity from a HA secondary node through VPN on primary node

[crlt]

I have two HA/CARP opnsense setups connected through a site to site Wireguard VPN. Since the Wireguard tunnel is only up on the active firewall, I cannot access the tunnel from the secondary/backup node. If I try to initiate a ping from the backup node at one site to the primary node at the other site, it does not work (ping: sendto: No route to host).

What is the best way to address this? Last I checked we couldn't assign a virtual IP to a wireguard interface.

I have an outbound rule so that VPN clients can access the backup node but I'm not sure how to go about it when the traffic is initiated at the backup node itself.

[bimbar]

This might best be solved by dynamic routing.

[crlt]

Any pointers on what such a rule would look like and how to get started? I'm not familiar with dynamic routing and it sounds like quite a lift when my main purpose is to just get logging/backups from the firewalls to opposite sites.

If IPSEC supports binding to a CARP VIP, would it just be simpler to move my site to site tunnels from wireguard to IPSEC?

[od1n0]

Check out the FFR plugins that support dynamic routing

[crlt]

I recently setup iBGP for some internal services so I thought I would attempt this with eBGP between my two HA opnsense nodes. In the end I was able to achieve this with active/active BGP on each router (each having a unique router-id). However there seems to be an issue (bug? expected?) during failover and/or maintenance mode (mainly happen when one router is put into maintenance mode but not always) where an erroneous route is installed which not only breaks routing between sites but sends the traffic out of the WAN interface. The only way to fix it is to stop FRR and start it (restarting does not fix it).  I suspect the cause is that routes are added before the wireguard site-to-site tunnel is ready.

This is the output in the FRR routing table. The second entry is supposed to be the site-to-site wireguard interface with it's tunnel address.

CODE NETWORK ADMIN DISTANCE METRIC INTERFACE INTERFACE_NAME VIA
B>* 10.20.10.0/24 20 0 <blank> <blank> 192.168.20.251
B>* 10.20.10.0/24 20 0 igb1 wan01 <WAN-IP>

After multiple steps to troubleshoot I gave up and figured that the potential for unexpected behavior during failover/maintenance was not worth it and eventually reverted back. Active/backup BGP does not solve it since the FRR daemon does not run on the backup I cannot reach the services on the site like I originally sought out to do.

[crlt]

I suspect the cause is that routes are added before the wireguard site-to-site tunnel is ready.

Actually now I believe the cause is something to do with the failed node giving an improper route because this does not happen when FRR is set to only be active on the master node and even then wireguard tunnel usually takes a minute to come back up.

Edit: Now I believe the issue may have been because I had an allowed IPs subnet in my Wireguard peer (from the site to site tunnel) that overlapped with the learned routes in FRR/BGP... Haven't had the chance to test but if anyone is doing something similar and finds this thread, try that. I will report back when I test.