[SOLVED] VPN : IPsec : IPv6 : Roadwarrior : Connect works but no traffic

[rainerle]

Hi,
we are currently adding IPv6 to our system setup as more and more of our users are upgrading their cable network data rates and are forced DS-Lite upon.

I went through the following steps:
- Add IPv6 to Interfaces and Virtual IPs
- Add IPv6 to IPsec server settings, pools and tunneled networks
- Add IPv6 IPsec pool addresses to firewall aliases, so the rules continue to work

Now I am facing the following problem:
I override the IPv4 address of the VPN DNS FQDN using the hosts file and try to connect via IPv6 to the VPN service. The connection gets established, the route print command looks fine but no traffic is passing. I am not able to ping IPv4 and IPv6 addresses behind the VPN.
When connecting to the VPN with the IPv4 address it works as expected, I am able to connect to services behind the VPN using IPv4 and IPv6 addresses.

The only obvious log entry difference is this:

Code: [Select]

Aug 21 22:34:15 opnsense01 charon: 05[IKE] <mobile-ops|6> peer requested virtual IP %any
Aug 21 22:34:15 opnsense01 charon: 05[CFG] <mobile-ops|6> reassigning offline lease to 'user'
Aug 21 22:34:15 opnsense01 charon: 05[IKE] <mobile-ops|6> assigning virtual IP 10.20.35.33 to peer 'user'
Aug 21 22:34:15 opnsense01 charon: 05[IKE] <mobile-ops|6> peer requested virtual IP 1:2:3:8001::1
Aug 21 22:34:15 opnsense01 charon: 05[CFG] <mobile-ops|6> reassigning offline lease to 'user'
Aug 21 22:34:15 opnsense01 charon: 05[IKE] <mobile-ops|6> assigning virtual IP 1:2:3:8001::1 to peer 'user'
Aug 21 22:34:15 opnsense01 charon: 05[CFG] <mobile-ops|6> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
Aug 21 22:34:15 opnsense01 charon: 05[KNL] <mobile-ops|6> adding PF_ROUTE route failed: Invalid argument
Aug 21 22:34:15 opnsense01 charon: 05[KNL] <mobile-ops|6> installing route failed: 10.20.35.33/32 via 1:2:3::1 src 10.11.10.11 dev ixl1
Aug 21 22:34:15 opnsense01 charon: 05[KNL] <mobile-ops|6> adding PF_ROUTE route failed: Invalid argument
Aug 21 22:34:15 opnsense01 charon: 05[KNL] <mobile-ops|6> installing route failed: 10.20.35.33/32 via 1:2:3::1 src 192.168.0.11 dev ixl1

Since IPv6 and IPv4 via the IPv4 VPN server address work but not via the IPv6 VPN server address I believe I am on the right track, but have no idea currently where to look further.

Please help
Rainer

[rainerle]

Looking at the tcpdump results I can see that a ICMP echo request from the road warrior and the resulting reply are created.

Code: [Select]

root@opnsense01:~ # tcpdump -n -v -i enc0 host 10.20.35.33
tcpdump: listening on enc0, link-type ENC (OpenBSD encapsulated IP), capture size 262144 bytes
02:45:25.413828 (authentic,confidential): SPI 0xc2297bc2: IP (tos 0x0, ttl 128, id 17883, offset 0, flags [none], proto ICMP (1), length 60)
    10.20.35.33 > 10.20.30.66: ICMP echo request, id 1, seq 490, length 40
02:45:25.414166 (authentic,confidential): SPI 0x3e6cdbde: IP (tos 0x0, ttl 63, id 22826, offset 0, flags [none], proto ICMP (1), length 60, bad cksum cc0c (->cd0c)!)
    10.20.30.66 > 10.20.35.33: ICMP echo reply, id 1, seq 490, length 40

But the reply does not arrive at the road warrior - I get a PING timeout...

So the way into the VPN is working, the route back to the road warrior seems broken.

[rainerle]

This seems to be related to:
https://forum.opnsense.org/index.php?topic=3957.0

Is there anybody who has a working IPv4 Tunnel over an IPv6 VPN network?

[rainerle]

Looking through the legacy configurations at
https://www.strongswan.org/testing/testresults/ipv6-stroke/
it seems IPv6 VPN connections need the

Code: [Select]

leftfirewall=yes setting on the firewall.

Has anybody tried that yet?

[rainerle]

So setting manually

leftfirewall=yes

using a include.d file did not help.

I created screenshots of the setup now now and maybe somebody else can try to reproduce the error (see attachment).

I also had a look into /tmp/rules.debug and here is the debug output

Code: [Select]

#debug:Interface any not found
# pass out on ##any## proto udp from {any} to {any} port {500} keep state label "1abfe6cf4f61a1db48bfc5b6dfb138cd" # IPsec: Default mobile client
#debug:Interface any not found
# pass in on ##any## proto udp from {any} to {any} port {500} keep state label "5e95c3fc98a6d9c9e5a550f8eabb544e" # IPsec: Default mobile client
#debug:Interface any not found
# pass out on ##any## proto udp from {any} to {any} port {4500} keep state label "29c33f8814fa348a17e4698a8ba88685" # IPsec: Default mobile client
#debug:Interface any not found
# pass in on ##any## proto udp from {any} to {any} port {4500} keep state label "90c77ae9faaeb85cb6e130171200ccc7" # IPsec: Default mobile client
#debug:Interface any not found
# pass out on ##any## proto esp from {any} to {any} keep state label "dbde0699f0974408534ce69e31300fbc" # IPsec: Default mobile client
#debug:Interface any not found
# pass in on ##any## proto esp from {any} to {any} keep state label "4e5377a181d8625177f31997261f4058" # IPsec: Default mobile client

Yes, I created the rules manually.

No idea how to continue from here now...

[rainerle]

Found the problem:

Code: [Select]

Aug 26 02:03:54 opnsense02 charon: 00[DMN] Starting IKE charon daemon (strongSwan 5.8.4, FreeBSD 11.2-RELEASE-p20-HBSD, amd64)
Aug 26 02:03:54 opnsense02 charon: 00[KNL] unable to set UDP_ENCAP: Invalid argument
Aug 26 02:03:54 opnsense02 charon: 00[NET] enabling UDP decapsulation for IPv6 on port 4500 failed

From that output I went here:
https://wiki.strongswan.org/issues/939

And from there I went here:
https://wiki.strongswan.org/issues/2599

And from there I found out, why I did see packets coming into the Firewall from Windows and MacOS and iOS devices, but Linux clients just gave an error.
https://docs.microsoft.com/en-us/windows-hardware/drivers/network/traversing-nats-and-napts-with-udp-encapsulated-esp-packets
Windows, MacOS and iOS can do UDP encapsulation for IPv4 and IPv6, Linux and OPNsense can only for IPv4. Therefore the packets never got back to the client from the OPnsense - hence Time out.

And then I changed the setting "NAT Traversal" on phase 1 from "Force" to "Enable" and IPsec mobile client VPN finally works.