Issues with VPN to LAN port forwarding

[vala4i]

I'm running quite basic configuration where Opnsense is connected directly to my ISP and to LAN switch. Opnsense has VPN client running (wireguard) which is connected to the server deployed in the cloud (self-hosted, IP 10.20.10.1). The goal is to access some LAN resources behind Opnsense through VPN to LAN port forwarding.

Lab environment:
 * Opnsense (VPN IP: 10.20.10.2)
 * Laptop (VPN IP: 10.20.10.8 ) — connected to the same VPN server as Opnsense.
 * Nginx in LAN is listening on 192.168.40.100:5001.

Opnsense firewall is configured in a way that 10.20.10.8:5001 redirects to 192.168.40.100:5001.
So, expected behavior is when laptop hits 10.20.10.8:5001 it being redirected to 192.168.40.100:5001. I guess this is a common setup for anyone who would like to access any LAN services over the VPN from external networks.

I'm running tcpdump on the laptop (VPN interface), Opnsense (VPN interface) and LAN server (on the interface where Nginx is listening).

I see that request from the laptop reaches Nginx, so apparently port forwarding works. But Nginx response never reaches the laptop. This is how traceroute looks from Nginx to the laptop:

traceroute to 10.20.10.8 (10.20.10.8 ), 30 hops max, 60 byte packets
 1  192.168.40.1 (192.168.40.1)  0.202 ms  0.266 ms  0.162 ms
 2  10.20.10.1 (10.20.10.1)  16.096 ms  16.075 ms  16.067 ms

I'm not sure if this is right that the first hop is 192.168.40.1, I'd expect that it would be 10.20.10.1. And running tcpdump on Opnsense I indeed see that 192.168.40.1 receives the response packet from Nginx but it never forwards it to the VPN server (10.20.10.1). Obviously that's the reason why the laptop doesn't get a response too.

Is that something that potentially can be configured using Opnsense (I mean VPN to LAN port forwarding)? What am I missing here?

[vala4i]

And forgot to mention that Nginx host actually can ping the laptop over VPN, though ping occasionally indicates redirect host:

PING 10.20.10.8 (10.20.10.8 ) 56(84) bytes of data.
64 bytes from 10.20.10.8: icmp_seq=1 ttl=62 time=87.8 ms
64 bytes from 10.20.10.8: icmp_seq=2 ttl=62 time=110 ms
64 bytes from 10.20.10.8: icmp_seq=3 ttl=62 time=33.1 ms
From 10.20.10.1: icmp_seq=4 Redirect Host(New nexthop: 10.20.10.8 )
64 bytes from 10.20.10.8: icmp_seq=4 ttl=62 time=52.4 ms
64 bytes from 10.20.10.8: icmp_seq=5 ttl=62 time=128 ms
64 bytes from 10.20.10.8: icmp_seq=6 ttl=62 time=21.8 ms
64 bytes from 10.20.10.8: icmp_seq=7 ttl=62 time=163 ms
64 bytes from 10.20.10.8: icmp_seq=8 ttl=62 time=41.8 ms
64 bytes from 10.20.10.8: icmp_seq=9 ttl=62 time=65.2 ms
From 10.20.10.1: icmp_seq=10 Redirect Host(New nexthop: 10.20.10.8 )
64 bytes from 10.20.10.8: icmp_seq=10 ttl=62 time=87.1 ms
64 bytes from 10.20.10.8: icmp_seq=11 ttl=62 time=110 ms

[vala4i]

OK, seems like my problem is related to this https://github.com/opnsense/core/issues/4389.
And Wireguard VPN to LAN port forwarding is just totally broken at this moment 

[Fright]

in a way that 10.20.10.8:5001 redirects to 192.168.40.100:5001

you mean 10.20.10.2:5001?

2  10.20.10.1 (10.20.10.1)  16.096 ms  16.075 ms  16.067 ms

is Wireguard server allows client-to-client traffic?

can you post your firewall rule (associated with port forward rule) from firewall-diagnostics-pfinfo-rules?

[Fright]

also..i dont use wireguard. but simple logic dictates that it should be site-to-site vpn between opnsense and cloud-server. and there will be no need to translate - just route. why not to use this method?

[Greelan]

I've just implemented WG in place of my previous OpenVPN server (which I ran on a local server, not on OPNsense). So I am not an expert, but I have figured out the fundamentals. 

I don't have a three-peer setup like you (my peering relationships are one-to-one), but if I (say) want to access a webserver on my LAN while connected to the VPN on my phone remotely, I simply browse to the webserver IP (or more accurately it's local domain) - no portforwarding involved.

Unless I am misunderstanding your use case, the key in my mind is that there needs to be a WG peering relationship between your laptop and OPNsense, and the laptop needs to have your nginx LAN IP as an AllowedIP. Then there needs to be firewall rules to allow the laptop WG IP to access the nginx LAN IP.

[Greelan]

You might find this tutorial (https://www.zahradnik.io/wireguard-a-vpn-with-real-world-usage-in-mind) useful to adapt to your circumstances. Specifically scenario #2, if you need to access OPNsense over the VPN from your laptop indirectly via the cloud peer. Although your OPNsense peer is presumably directly accessible from your laptop over its public IP?

As the tutorial notes, the fundamental shift in thinking when moving from OpenVPN to WG is that WG essentially does not operate on a server-client model, but rather a peer-peer model (although in practice some setups might look like a server-client model).

[vala4i]

@Greelan thanks for the input.

I guess site-to-site setup might solve the problem. I remember how I tried to configure it with OpenVPN many years ago and failed. Since that I've been using port forwarding to access LAN resources over VPN (before with OpenVPN and now with Wireguard using Unifi USG but planning to move to Opnsense).

I've tried to play with AllowedIPs on both clients and "server" (I know that all peers are equal) but without any luck. Either I'm doing it wrong or something else is needed like additional NAT or firewall rules.

So here is my server config (has public WAN IP, deployed on VPS in AWS):

[Interface]
PrivateKey = <private-key>
ListenPort = 51820

[Peer]
# Opnsense
PublicKey = T48KrajQsS0hfY7e3A0df3zLi9ODX+naSH3STq7dpjw=
PresharedKey = <pre-shared-key>
AllowedIPs = 10.20.10.2/32

[Peer]
# Laptop
PublicKey = R534tXr0h1UapigeobiPt7h2yiNqce/LNj/QkkVq7Ws=
PresharedKey = <pre-shared-key>
AllowedIPs = 10.20.10.8/32

There're more peers there (other laptops, cell phones etc.) but I guess that doesn't matter for now.

Laptop configuration (it can be in any network, but I expect that as soon as I turn on Wireguard client, I get access to my home LAN resources behind Opnsense):

[Interface]
PrivateKey = <private-key>
Address = 10.20.10.8/24
DNS = 10.20.10.3

[Peer]
PublicKey = sqpFAEdKQ4kCoUvo00dK2ZIeAOWNvk0WAmTVFHJtBQk=
PresharedKey = <pre-shared-key>
AllowedIPs = 0.0.0.0/0
Endpoint = <server-endpoint:port>
PersistentKeepalive = 25

Opnsense configuration is attached (it acts as my home router with WAN connected to ISP and LAN to local switch).

And LAN subnet where web server is deployed is 192.168.40.0/24.

Based on the blog article above, I realize that this LAN subnets needs to be added to AllowedIPs list and I tried to do that but failed. So I'd appreciate if someone could point me to the changes I need to make here.

[vala4i]

@Fright appreciate your input too. As I said above, site-to-site VPN sounds like a plan. But I'm a bit unclear regarding what changes I need to do in order to make it work as expected 

[Greelan]

Again, I don’t have this setup where I am only indirectly accessing one peer via another peer. But a couple of suggestions.

Untick “Disable Routes” on the local configuration on your OPNsense router. Also, try adding the LAN IPs to the OPNsense peer configuration on the cloud server (in addition to 10.20.10.2/32) - my thinking here is that without that, the cloud server does not know that packets to the LAN IP that are received from the laptop need to go via the tunnel to the OPNsense router.

But is there a reason that you can’t have a direct peer relationship between the laptop and OPNsense router? Doesn’t the OPNsense router have a public IP that you can use as the endpoint for a peer configuration on the laptop? If so, add a peer for the OPNsense router on the laptop, and a peer for the laptop on the OPNsense router. You will then need to adjust the AllowedIPs for the cloud peer config so that it doesn’t include the laptop VPN IP, otherwise you will be telling WG on OPNsense that the same IP is provided by two peers. Same on the laptop - you will need to adjust which AllowedIPs go via the tunnel to the cloud server, and which go to the OPNsense router.

[Greelan]

BTW, replacing a USG with an OPNsense box is also the move I made a few weeks ago. So glad I did.

[vala4i]

I made it work finally. Basically the key thing is to have a route to LAN on the server which sits in the middle. I had to add LAN subnet to AllowedIPs there and also manually added the route (I guess wg-quick can do that automatically or via PostUp directive but I'm not using it).
And on laptop's side traffic to LAN should be routed through the tunnel as well.

I haven't tested performance yet and probably will need to play with MTU to make it work smoothly but the main thing is that it works and I don't care about port forwarding bug anymore

Thanks a lot for the idea with site-to-site VPN setup.

[Greelan]

Great stuff. Yes, my understanding is that out of the box WG creates the tunnel routes for you so that’s why I assumed you would only need to add the LAN as AllowedIPs on the cloud server for the OPNsense peer.

Anyway, glad you got it to work!