Policy Based Routing over IPSec VPN

[leacho73]

Hi All,

Sorry to bring up an old topic which I started in Feb 21 with regard to Port Forwarding over a IPSec VPN connection - but I'm still having no luck in getting it working. (Original Post here: https://forum.opnsense.org/index.php?topic=21347)

Basically what I am trying to achieve is to port forward from an external facing IP in Site A to a server in Site B - to deliver a web page. The difficulity is that Site A and Site B are connected via an IPSec VPN and the server in Site B is connected to another OpnSense Box - the layout is similar to below:

WWW -> Site A IP -> Site A OpnSense -> IPSec to Site B -> Site B OpnSense -> Server

I currently have a policy based route setup on the network connected to the Site B OpnSense server - which says for all traffic in 10.0.0.0/16 subnet to come out of the gateway located on the Site A OpnSense side of the VPN - this is working, to a degree - I can ping internet traffic, I can browse bbc.co.uk via a web browser, but not google.co.uk - but when I run a DIG command, I can see that I am coming out of the correct IP on the Site A Side.

What I do see when attempting to Port Forward TCP/443 to the server in site B is that the connection makes it all the way to the server in Site B - but then I see a load of TCP Retransmissions via wireshark, which look like an routing issue.

All the firewall logs show green and passing the connection - so I am completely lost as to what I am missing now!! I've been looking at this for months with no solution.

If anyone has made this work in any setup, I'd love to hear from you - shout out to errored_out who helped on the last post - but unfortunately, I still couldn't get it going.

Thanks
Leacho

[kosta]

Hi,

I recently ran into an issue with TCP Retransmissions due to routing and it also had to do with IPsec tunnel.
Not quite the same configuration like yours, but similar enough.

The problem on my side that the two routes, going from remote server to the client and from the client to the remote server were different, and OPNsense, being a stateful firewall, had a problem with it. One route was going over the OPNsense, the other weren't. The issue is called "asymmetric routing".

Once I fixed the routing by creating a transfer network and appropriately and cleanly routing both ways over the same route, the problem disappeared.

Maybe think about that for a moment and see if that helps.

[leacho73]

Thanks for coming back to me Kosta - it sounds plausible - I think it definitely related to the IPSec VPN, as that's where I get the routing issue with the return traffic - I'll check the routing, but its all BGP, so 'shouldn't' be an issue but worth investigating.

Re: the transit network - how did you go about that may I ask?

Thanks
Leacho

[kosta]

Re: the transit network - how did you go about that may I ask?

In our case, it was a bit complicated, since the network on one side of the tunnel can only talk to a specific network on the other side. It's some restrictions that I have to live with and work around.

Basically it's like this:
A Cisco VPN router has an interface on the OPNsense ("WAN") over which it connects via IPsec to the remote address.
Then I created a LAN connection between OPNsense and the Cisco, which is basically a transfer network (nothing in there but two IP, one for OPNsense, other for the Cisco).
I had to change the local network to anything else, since we then created the a 1:1 NAT on the Cisco to translate the local network into the network that remote server can talk to.
And that's it basically.
On the OPNsense I only had to create static routes for the remote networks through the new transfer-network gateway (the IP is the one of the Cisco in the transfer-network, of course).
On Cisco we had to create routes to the local networks, too. Over OPNsense transfer-network IP for local network(s).
Additionally I have to create outbound NAT rules to NAT the networks into Cisco that have not been entered in the Cisco.

All somewhat complex, but it's working like a charm.