Disable reply-to on WAN rules not taken into account for auto-added VPN rules

[glasi]

Dear all,

I have a working IPsec VPN setup with multiple site2site and roadwarrior tunnels. Currently, I only use IPv4 for my IPsec connections. Works like a charm. To make my setup more future-proof I'm a currently testing IPv6 IPsec connections. This is exactly where the problems start...

To make a long story short: IPv6 IPsec connections cannot be established.

A packet capture shows that the OPNsense responds to incoming ISAKMP traffic. However, response packets never reach the IPsec originator. It looks like the IPv6 gateway drops the packets for whatever reason.

Further research in the forum put me on the right track. The problem seems to be related to reply-to which by default is added to every WAN rule.

Hence, I disabled reply-to on WAN rules under Firewall -> Settings -> Advanced. However, that did not bring the desired success. ISAKMP response packets still do not reach the IPsec originator.

I digged a little bit deeper, disabled all auto-added VPN rules and started from scratch with my own firewall rules. And lo and behold... it works!

In the end I could pinpoint the problem. Auto-added VPN rules do not take the global firewall setting "Disable reply-to on WAN rules" into account.

Despite disabling reply-to auto-added VPN rules are still being generated with a reply-to expression:

pass out log on pppoe0 route-to ( pppoe0 fe80::xxxx:xxxx:xxxx:xxxx ) proto udp from {any} to {any} port {500} keep state label "37[...]c7" # IPsec: Mobile Devices

pass in log on pppoe0 reply-to ( pppoe0 fe80::xxxx:xxxx:xxxx:xxxx ) proto udp from {any} to {any} port {500} keep state label "27[...]81" # IPsec: Mobile Devices

Unless someone provides a good reason why that is so, I have to assume that this is a bug.

[glasi]

Github ticket can be found here now: https://github.com/opnsense/core/issues/4942

[schnipp]

I observed the same behavior some time ago, but unfortunately had no time to investigate. I need to hurry up to get IPv6 IPsec VPN running. Currently, I own a dual stack internet connection with a public IPv4, but my ISP has begun to switch internet connections to dual stack lite with CGNAT  . Without a reliable IPv6 aware IPsec VPN I will run into trouble in the future, especially when using dial-in connections from mobile devices.

So, I resumed investigations regarding this issue and played around with the netcat (nc) utility in the Opnsense shell. Prior to this, I installed an apropriate firewall rule for incoming traffic on the WAN (PPPoE) interface with "reply-to" setting enabled. Regardless of using TCP or UDP the IPv6 communication from a client (nc in client mode) to the Opnsense (nc in server mode) is faulty. Using UDP messages can be sent from the client to the Opnsense host, but not in the other direction. Using TCP the 3-way handshake fails.

A packet dump confirmed that packets from the Opnsense host will not reach the client which initiated the network connection. After disabling the "reply-to" setting within the firewall rule I was very surprised that TCP and UDP connections work well. In general, the "reply-to" setting should not have any effect to the network communication, because I do not own a multi-wan environment. I analyzed packet dumps of both scenarios using wireshark and couldn't figure out what's the root cause, because both packet dumps look nearly the same .

The last idea which came to my mind was looking at the TCP and UDP checksums. And indeed, in the standard configuration (reply-to enabled) packets originated from the Opnsense host contain an invalid checksum. When disabling the "reply-to" setting the checksums are ok.

Although, the automatically generated VPN rules do not take the global "reply-to" setting into account this is not the root cause of faulty IPv6 connections. Rather, this setting probably triggers a bug in the network stack of FreeBSD.

I can't find a suitable report in Bugzilla (but a similar one).

Maybe anybody has an idea?