OPNsense Forum

English Forums => General Discussion => Topic started by: whatever32 on June 09, 2020, 06:15:42 am

Title: WireGuard - This is the strangest issue I've ever encountered...
Post by: whatever32 on June 09, 2020, 06:15:42 am
Hello,

Longtime reader, first time poster.

So I have an issue that I'm unable to make any sense of. I'll try and explain it as thoroughly as possible.

I initially encountered this issue on my "production" box, which has quite a few packages running. Thinking it may be related to my configuration, I took a spare computer and installed OPNsense on it form scratch. I'm using this system for all of my tests.

The system only has a WAN and a LAN. The WAN is directly connected to the modem. The only LAN client is my laptop, connected via Ethernet, and which is running a Linux Mint live image from USB. So the issue can't be related to the configuration of the laptop or the browser. The only plugin installed on OPNsense is WireGuard. Nothing else.

So I set the box up leaving almost everything at the default settings. All I did was disable IPv6 and the DNS Resolver, and I gave a system a public DNS server. Default firewall rules. All good. Everything is working. I can access the Internet.

Then I configure WireGuard, with iVPN as a provider, following this guide:

https://www.ivpn.net/setup/router-opnsense.html

The configuration works. I just need to assign the VPN's DNS server to my client, either manually or via DHCP, and I am connected to WireGuard. I can see the handshake and the sent and received data and everything. And I have Internet access. And my public IP is that of the VPN server. Running a DNS leak test shows me using iVPN's DNS server within the tunnel...

BUT a bunch of Https site break (reddit.com, github.com) and a bunch of services break (apple appstore, ps4 online gaming). Now, I understand that at this stage, this could be happening for a million different reasons:

-Wrong Time on the System: My system time is correct. Using NTP.
-DNS Misconfiguration: DNS works fine. I can "reach" the sites juste fine, but the TLS handshake never completes and the site never loads (I'm assuming it's the same issue for the services that break). Also, I can ping the sites in question just fine. No crazy latency, nothing. I also tried with different DNS servers and the same issue remains.
-MITM, such as a proxy: OPNsense is vanilla. No proxy running on OPNsense or on the client device.

I rebooted a million times. I tried different VPN servers. I tried different ports. I tried deleting and uploading a new key and getting a new IP. No dice. Same issue.

Then, as I was pulling my hair out, I decided to buy a month from a different provider, to test with: Mullvad. I uploaded a public key to Mullvad and was assigned an IP. I configured Mullvad exactly as I had configured iVPN and I was expecting it to fail. But to my surprise, it just worked. Everything just worked. The sites that wouldn't load on iVPN loaded and the services that broke on iVPN were working just fine.

Now, iVPN know what they're doing and I couldn't believe that their WireGuard service was broken. They would have gotten hundreds of support requests were that the case. So, this wasn't a resolution for me.

So, I purchased another month from another VPN provider that supports WireGUard: OVPN, this time. I again uploaded a public key and got an IP address. Then I configured WireGuard again with OVPN, exactly as I had with iVPN and Mullvad. And... It behaved exactly like iVPN did: broken Https sites and broken services.

Then I noticed something: The IP address that iVPN (and OVPN) assigns you when you upload your public key for WireGuard is always of the form 172.x.x.x. Mullvad, on the other hand, assign a 10.x.x.x IP address. I also use iVPN with OpenVPN. When using OpenVPN, they assign a 10.x.x.x IP address, which has always worked and continues to work.

So, I can only conclude that something on my system blorks up WireGUard when the interface's IP address is of the form 172.x.x.x. But if the interface IP address is of the form 10.x.x.x., everything works perfectly. But I cannot understand why this would be happening, so I have no idea where to look or what to try and fix. And again, this is an almost vanilla system. No plugins (aside from WireGuard). And I'm testing with my laptop running a Linux Mint live image from USB...

I'm just at a complete loss and was hoping someone on here might have an idea...

Cheers

Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: mimugmail on June 09, 2020, 07:22:13 am
Interfaces : LAN : MSS

Set this value to 1300 for testing. Maybe a frag issue
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: whatever32 on June 09, 2020, 08:11:31 am
That was it. I can't thank you enough. I've been struggling with this "bizarro-world" issue for weeks and setting the MSS value on the LAN to 1300 just did it.

If you can explain a little bit what was happening, I'd love to gain some insight.

Either way, thank you very much for the help.
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: mimugmail on June 09, 2020, 08:25:38 am
It seems the MTU is too low on wireguard and the ICMP information to your client sending to huge packets get's lost. With MSS you change this level for TCP packets on the interface so communication runs with lower size.
1300 usually works for all.
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: whatever32 on June 09, 2020, 08:32:39 am
So in the end, it had nothing to do with the assigned IP. It's the way the MTU is configured on the provider's side?
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: mimugmail on June 09, 2020, 08:36:59 am
On your side .. (the interface created on your firewall)
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: whatever32 on June 09, 2020, 08:39:32 am
But then why did Mullvad work without the MSS change?
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: mimugmail on June 09, 2020, 10:16:08 am
OK, then it seems they use a different MTU
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: whatever32 on June 09, 2020, 11:21:04 pm
Probably. But I get what you're saying: setting the MTU on my interface is the way get it to work with most providers, which could all be different.

Again, thanks a lot for your help.
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: slatesky284 on October 24, 2020, 01:53:18 pm
Wow I've been struggling with this for months! It finally works so thanks for sharing! I contacted ivpn support, described the exact issue you did, and they had no idea about this MSS setting.
Title: Re: WireGuard - This is the strangest issue I've ever encountered...
Post by: Gauss23 on October 24, 2020, 02:10:46 pm
You can try and raise the MSS again until it stops working.