[SOLVED] WireGuard ProtonVPN connection active, but unable to receive responses

[ctrom]

I have followed the documentation for setting up WireGuard with ProtonVPN. The VPN status indicates that the connection is online, the handshake age is being refreshed regularly, and there is data being sent and received, although the received traffic is about 1/4 of the sent traffic. I am able to ping the address Proton specified (10.2.0.2) but pings to 8.8.8.8 are lost and attempts to curl -v http://neverssl.com result in Recv failure: Connection reset by peer

I have tweaked many settings and spent a few hours going back and forth with Gemini trying to identify what's wrong, but have had no success. I'm hoping for suggestions on what I should try or how I can diagnose where the failure is occurring.

[vimage22]

Not sure if this applies, as I do not use ProtonVPN, but have you tried looking at Firewall: Log Files: Live View? It helped when I was setting up WireGuard. Turn on logging here: System: Settings: Logging and Firewall: Settings: Advanced.

[ctrom]

Not sure if this applies, as I do not use ProtonVPN, but have you tried looking at Firewall: Log Files: Live View? It helped when I was setting up WireGuard. Turn on logging here: System: Settings: Logging and Firewall: Settings: Advanced.

Yes, I have enabled logging on all of the firewall rules related to the VPN. When I look at Live View, I see many requests that are passing from local IPs out of the network, but nothing from outside coming in. I've also looked at the VPN logs and the system logs and I haven't seen anything that indicates to me a failure condition.

[ctrom]

After too many hours of pulling my hair out, I finally realized I had mistyped the IP address for the gateway I had set up for the VPN. I had typed 10.2.0.1 when it should have been 10.2.0.2. The health check is passing now for the VPN gateway and I see return traffic on the firewall logs. I am facing a new issue now with my attempts to ping 8.8.8.8 which is that the ping response from my router's IP address:

Code Select Expand

$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=1.42 ms (DIFFERENT ADDRESS!)
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=1.49 ms (DIFFERENT ADDRESS!)

If anyone knows offhand what I might have done wrong to cause this please let me know. In the meantime, I am happy to finally have a different problem to work on.

[vimage22]

One little typo, happens to me as well. Maybe my post here might give you ideas? (#8) https://forum.opnsense.org/index.php?msg=262436
But without a lot more info on your config, not sure I can figure this out. Might be a Rule issue?

[ctrom]

One little typo, happens to me as well. Maybe my post here might give you ideas? (#8) https://forum.opnsense.org/index.php?msg=262436
But without a lot more info on your config, not sure I can figure this out. Might be a Rule issue?

I've attempted to capture all of the relevant settings here. If I've overlooked something please let me know.

WireGuard settings:
Instance:
Listen port: 51820
MTU: 1420
DNS Servers: 10.2.0.1
Tunnel address: 10.2.0.2/32
Disable routes: yes
Gateway: 10.2.0.1 - this is not specified in the ProtonVPN connection details, but the OPNsense setup documentation indicates this should be set.

Peer:
Allowed IPs: 0.0.0.0/0
Endpoint address: 79.127.136.222
Endpoint port: 51820

Interfaces:
WAN_ProtonVPN:
Device: wg0
IPv4: None

VPNOnly:
Device: vlan0.50
IPv4: Static
Address: 10.12.50.1/24

Gateway:
Name: ProtonVPN
Interface: WAN_ProtonVPN
IP Address: 10.2.0.2 - I'm unsure about this setting. I had it set to 10.2.0.1 and the gateway was reporting offline. After changing it to 10.2.0.2 the gateway reported online, but Gemini is insisting that this is inaccurate because it is creating a loop within the router.

Firewall:
NAT Outbound:
Mode: Hybrid
Custom Rule:
Interface: WAN_ProtonVPN
Source address: 10.12.50.0/24
Translation / target: Interface address
Static-port: yes

Aliases:
WG_VPN_Hosts: 10.12.50.1/24

Rules:
I have tried many different rules at this point including:

ID	Interface	Quick	Action	Direction	Source	Destination	Gateway	Advanced
1	Any	No	Pass	Out	WAN_ProtonVPN address	(invert) WAN_ProtonVPN net	ProtonVPN	Allow options:1 Disable reply-to:1
2	VPNOnly	Yes	Pass	In	VPNOnly net	RFC1918_Networks	None
3	VPNOnly	Yes	Pass	In	WG_VPN_Hosts	10.2.0.1	ProtonVPN	Set local tag: NO_WAN_EGRESS
4	VPNOnly	Yes	Pass	In	WG_VPN_Hosts	(invert) RFC1918_Networks	ProtonVPN	Set local tag: NO_WAN_EGRESS
5	WAN	Yes	Block	Out	any	any	None	Match local tag: NO_WAN_EGRESS
6	VPNOnly	Yes	Pass	In	any	any	ProtonVPN	Disable reply-to:1
7	WAN_ProtonVPN	No	Pass	In	any	any	None

These rules haven't been active all at once and there are many variations on these that I have tried in my experimentation.

Normalization
Interface: WAN_ProtonVPN
Direction: In
Max mss: 1300

This normalization setting isn't mentioned in the setup guide I followed, but was recommended by Gemini. I'm skeptical that the issue is related to packet size as the ping packets I've been generating are much less than 1,000 bytes.

[vimage22]

OK, give me some time to look at this.

[DEC740airp414user]

did you set the MTU on the interface you created for the wireguard tunnel?   

1400 seems to work good for my needs.

then  "Go to Firewall ‣ Settings ‣ Normalization and add a new rule to prevent fragmentation of traffic going through the wireguard tunnel." https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html   clamp MSS to 1360

[ctrom]

did you set the MTU on the interface you created for the wireguard tunnel? 

1400 seems to work good for my needs.

then  "Go to Firewall ‣ Settings ‣ Normalization and add a new rule to prevent fragmentation of traffic going through the wireguard tunnel." https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html  clamp MSS to 1360

I have edited my earlier reply with my configuration details to include my current settings for normalization.

[DEC740airp414user]

did you set the MTU on the interface you created for the wireguard tunnel? 

1400 seems to work good for my needs.

then  "Go to Firewall ‣ Settings ‣ Normalization and add a new rule to prevent fragmentation of traffic going through the wireguard tunnel." https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html  clamp MSS to 1360

I have edited my earlier reply with my configuration details to include my current settings for normalization.

opnsense manual says mss needs to be 40 below MTU.
yours seems way too low

better link:  https://docs.opnsense.org/manual/how-tos/wireguard-client.html

[ctrom]

opnsense manual says mss needs to be 40 below MTU.
yours seems way too low

better link:  https://docs.opnsense.org/manual/how-tos/wireguard-client.html

I started with a higher value and decreased it as part of my experimentation. Setting it to 1380 or 1360 doesn't improve the behavior. I was under the impression that setting a lower max mss would reduce network performance, but otherwise not affect the behavior of the packets, so even if it was lower than necessary, I should be able to get packets across the network. Is that inaccurate?

[ctrom]

I have been trying to gather more information about what my router is actually doing with the traffic. If I capture the UDP packets on my WAN interface that are associated with the WireGuard instance listen port, I see a lot of traffic in both directions:

Code Select Expand

# tcpdump -ni igc0 udp port 51820
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igc0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
01:18:28.926918 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:28.938407 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:28.940052 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:28.942216 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 1040
01:18:28.942226 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 1040
01:18:28.942237 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:28.953413 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:28.953423 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:28.953449 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:29.127551 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:29.139444 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:29.141381 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:29.141395 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 1040
01:18:29.141429 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 672
01:18:29.152319 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:29.152328 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:29.159271 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
01:18:29.161419 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:29.161428 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:29.162871 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 96
01:18:29.172365 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 96
This was captured in 0.25 seconds.

But if I target the VPN endpoint IP, I see much less traffic:

Code Select Expand

# tcpdump -ni igc0 host 79.127.136.222
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igc0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
00:50:04.910854 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:50:29.939964 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:50:54.978966 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:50:54.979594 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 148
00:50:54.990808 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 92
00:50:54.991716 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:51:20.016215 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:51:45.093979 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:52:10.152227 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:52:35.229948 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:53:00.243689 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:53:00.243882 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 148
00:53:00.255903 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 92
00:53:00.256202 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:53:25.296453 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:53:50.320640 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:54:15.326214 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:54:40.370420 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:55:05.393058 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 32
00:55:05.393689 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 148
00:55:05.415132 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 92
This was captured in 5 minutes.

I don't know enough about the behavior of tcpdump to understand why both packet captures have the VPN endpoint IP in them, but targeting the port reveals much more traffic than targeting the host.

If I do a packet capture targeting the host while attempting to ping an external IP:

Code Select Expand

# tcpdump -ni igc0 host 79.127.136.222
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igc0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
00:17:27.234282 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:28.234361 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:29.234570 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:30.234764 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:31.234854 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:32.235018 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:33.235196 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:34.235373 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:35.235554 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:36.235744 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:37.235794 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:37.398617 IP 79.127.136.222.51820 > {WAN IP redacted}.51820: UDP, length 32
00:17:38.235959 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:39.236117 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:40.236305 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:41.236453 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:42.236587 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
00:17:43.236779 IP {WAN IP redacted}.51820 > 79.127.136.222.51820: UDP, length 128
I see the pings are transmitted from the WAN IP to the VPN endpoint at a rate of one per second, but I don't see any corresponding responses.

I have two questions that come from this:

• Is my tcpdump command correctly filtering packets in a way that should be showing the responses to my pings?
• Would tcpdump reveal traffic that is hitting the WAN interface but is blocked by the firewall?

If the packet logging occurs prior to the firewall, I would expect to see response packets, which might indicate my packets are malformed leaving my network. Maybe a NAT issue?
If the packet logging occurs after the firewall, I think that would indicate my firewall rules are wrong.

[FredFresh]

If you disable the 8.8.8.8 ping, does the wireguard connection work?

Why did you block anything goign out from Wan? Can you disable that rule and try again if it works?

[ctrom]

If you disable the 8.8.8.8 ping, does the wireguard connection work?

Why did you block anything goign out from Wan? Can you disable that rule and try again if it works?

The 8.8.8.8 ping is not ongoing and when I stop the ping I am still unable to curl or otherwise access resources through the VPN.

The WAN block rule is disabled and has been since early on in my testing. I originally added the "NO_WAN_EGRESS" rules because they were recommended in the setup documentation.

Since I can see the ping packets leaving encrypted on the WAN interface, I'm fairly confident that the routing is working going out on the VPN. This leaves me with 2 theories:

• The packets that are leaving my network are malformed in a way that causes no return traffic to be generated.
• The rules I have in place are blocking the response somehow. I have enabled logging on every rule related to the VPN and I don't see anything in the live view that indicates something is being blocked, but I'm not sure I can rule it out.

[FredFresh]

Try to go here and check if returns the proton public ip or the ip of your ISP: dnsleaktest.com.

You monitor the wan interface, younshall consider that it is a phisical interface and the wireguard works "inside that"...you should see the same message going outside on both gateways and Not only on the wan.