wireguard 1:1 nat routes back over wrong gateway

[Voodoo]

Ive setup a wireguard tunnel for 1:1 nat.

Outbound nat is working fine, but port forwarding from the public ip to local client is not.

The local client receives SYN packets and answers with ACK, but opnsense is sending the ACK over the default wan gateway which drops them. I dont get why, this should be a stateful connection and opnsense should route them back how they came over the vpn. Also for the vpn interface (10.127.127.1) the gateway (10.127.127.2) is set, so it should use it, i dont get why it routes over wan.

I tried checking "Disable reply-to on WAN rules" but it has no effect.

note: port 55555 instead of 80 is used to easily filter it in captures
local client: 192.168.1.50, http server on port 55555
opnsense vpn: 10.127.127.1
gateway: 10.127.127.2 (public ip 45.157.xxx.xxx)

1:1 nat between 45.157.xxx.xxx and 10.127.127.1
1:1 nat between 10.127.127.1 and 192.168.1.50

trying a netcat from 207.246.xxx.xxx to 45.157.xxx.xxx:55555 results in the following capture dump:

Code: [Select]

state dump:
all tcp 192.168.1.50:55555 (10.127.127.1:55555) <- 207.246.xxx.xxx:45046 SYN_SENT:ESTABLISHED

capture:
Interface Capture output
lan
vtnet0 20:52:20.631158 IP (tos 0x0, ttl 52, id 11465, offset 0, flags [DF], proto TCP (6), length 60)
    207.246.xxx.xxx.45046 > 192.168.1.50.55555: Flags [S], cksum 0xb68a (correct), seq 1839892123, win 64240, options [mss 1460,sackOK,TS val 3739768784 ecr 0,nop,wscale 6], length 0
    192.168.1.50.55555 > 207.246.xxx.xxx.45046: Flags [S.], cksum 0xa05b (correct), seq 3704964607, ack 1839892124, win 65280, options [mss 1372,sackOK,TS val 2707705387 ecr 3739768784,nop,wscale 7], length 0   
    # removed duplicates / retries

wg1
wg1
    207.246.xxx.xxx.45046 > 10.127.127.1.55555: Flags [S], cksum 0xeee4 (correct), seq 1839892123, win 64240, options [mss 1460,sackOK,TS val 3739768784 ecr 0,nop,wscale 6], length 0
    # removed duplicates / retries

wan
vtnet0_vlan4
    10.127.127.1.55555 > 207.246.xxx.xxx.45046: Flags [S.], cksum 0xd8b5 (correct), seq 3704964607, ack 1839892124, win 65280, options [mss 1372,sackOK,TS val 2707705387 ecr 3739768784,nop,wscale 7], length 0
    # removed duplicates / retries


wg1 interface overview shows the gateway is set:

Code: [Select]

wg1 interface (opt4, wg1)
Status up
MAC address 00:00:00:00:00:00 - XEROX CORPORATION
MTU 1420
IPv4 address 10.127.127.1 / 30
Gateway IPv4 10.127.127.2


The gateway shouldnt even matter, 1:1 nat should route back as is. and if that doesnt work at least use the gateway set ? 192.168.1.50 also has a pbr to route over 10.127.127.2. There should be no way for traffic to hit wan.

So why the hell is opnsense routing over wan and not wg1 ? did i miss something ?

[5SpeedFun]

Do you have a network diagram available?

Feel free to obscure the ips...

[sashxp]

I think, you have the same Problem as i have it discribed here: https://forum.opnsense.org/index.php?topic=17973.0

and here is another guy with a similar Problem: https://forum.opnsense.org/index.php?topic=18013.0

mimugmail said, that there is no expectation for a fast Solution. :-(

[Voodoo]

Thanks for the links, i missed them searching the forum.

I tried setting more pbr to route over the gateway but they had no effect.

Whats interesting is, if i set a address translation on the wireguard gateway to rewrite incoming ip's to the gateway ip 10.127.127.2, everything works. The only issue of course my local client only sees the gateway ip which i really would like not to be the case. But i just cant get the routing to work with wireguard.

Openvpn works, so it must be some issue with wireguard. Or im missing something.