Problem accessing webgui with two CARPed OPNsense firewalls

[clopmz]

Good morning,

I have configured two OPNsense firewalls in HA mode using CARP. These firewalls connect via IPsec tunnel to CheckPoint firewalls. Everything works correctly: the hosts behind the OPNsense firewalls are reachable from the hosts behind the Checkpoint firewalls and viceversa, etc.

But the problem comes when I try to manage these OPNsense firewalls from a computer behind CheckPoint firewalls. I can always access the master node but never the backup node (neither by ssh nor by webgui.). If I shut down the master node, I can access the node that was backup without any problems both via ssh and webgui.

I have enabled "Disable reply-to on WAN rules" but nothing. Analysing the traffic that arrives to the node that I cannot access, I can see how the ssh and webgui requests arrive and do so through the IPsec tunnel, but do not return.

What could be the problem with the backup node, is it a routing problem through the IPsec tunnel?

Many thanks for your help.

[Monviech (Cedrik)]

If you can SSH to the master node, and you can SSH from the master to the backup node, you could forward the OPNsense Website port through the SSH tunnel. 

[clopmz]

Many thnaks Monviech. I had thought about the same thing, but it is not very operative ...

But I can't understand why, having both firewalls configured the same, I can access one and not the other.

Another option that exists is to set up a WireGuard tunnel in each one and I think it will work but it complicates maintenance a lot ...

The thing is that I can't see what the problem is ...

[Monviech (Cedrik)]

Well you are probably trying to reach the CARP VIP of the firewall, it is always tethered to the Firewall that is the current master.

What if you try to reach the actual interface addresses of your backup firewall?

[clopmz]

When I access both by ssh and https I do it to the real IP of the fw, never to the CARP virtual ip.

But, the result is always the same: I can not connect to backup node until I stop master node.

[Monviech (Cedrik)]

What does the firewall log of the backup node say? Maybe there is a missing firewall rule that allows it?

[clopmz]

I don't think so... In the firewall logs there is nothing, no blocking ... Running a tcpdump on the pflog0 interface of the backup firewall.

If I run a tcpdump on the LAN interface of the backup fw I see packets arriving but not returning ...

[Monviech (Cedrik)]

I guess this would need a layer 3 network diagram. Maybe there is some sort of routing issue somewhere, since there are IPsec tunnels in play (didn't say if policy based or VTI).

I guess it could take some effort to troubleshoot this.

[clopmz]

My IPsec tunnels are policy based ...

Uhmm ... It is very rare ...