Wireguard over Mullvad (VPN) does not work anymore after upgrade to 23.7.4

Started by guest35379, September 16, 2023, 12:46:18 PM

Previous topic - Next topic
Hi,

I previously followed the instructions in the official documentation to let all traffic from both the firewall as well as all clients exit through a wireguard VPN with Mullvad. Of the two available plugins for wireguard I used the version integrated into the kernel.

I setup the connection in the plugin, created an interface for the wireguard connection and setup an outbound NAT rule. This was originally very easy to setup and worked on the first try until I did the most recent update. After the update and a subsequent reboot the connection stopped working and I have been unable to set it up again. At first I though it might be a DNS problem, but neither the firewall nor the clients can even ping any outside address.

I really hope someone with more experience can tell what has changed (I found nothing helpful in the release notes) or what I need to change in this setup to make it work.

Thank you.

Check server status. If you touched your previously working configuration it's better to start over. There are no issues in 23.7.4 to be worried about here.

https://mullvad.net/en/servers

I use Mullvad as well and am assuming you're running into the same issue as me. If so... It's actually the upgrade to 23.7.3 that killed it, but OPNsense didn't reboot during that version like it did for 23.7.4, so it was kind of broken without knowing.

I got my VPN itself working via my latest post here: https://forum.opnsense.org/index.php?topic=35972.0
However, I'm still running into the issue where the firewall itself is not being routed through the VPN. This is likely due to the route that I deleted, mentioned at the end, but I'm not sure. I've got a separate thread here, where I'm trying to figure this out still: https://forum.opnsense.org/index.php?topic=35977.0

Note that the solution to the first link is probably mostly a hacky-version of it. TL;DR to it: I followed the Mullvad instructions from the pfsense+wireguard, but also checked the "disable routes" checkbox in the wireguard server settings (So it'd work on reboot).

Basically, it appears that when wireguard got moved from a software module over to the kernel, it took the changes for gateway configuration OR the routing table with it, resulting in it not working (i.e. killswitch was being hit so VPN "didn't work")... That's my current theory anyway. Perhaps the `0.0.0.0/0` allowed IPs is not a solution anymore for opnsense? No idea, lol. Just a bunch of messy theories I've got going in my head at the moment.

Hello There,
I am also one of those using wireguard vpn with mullvad. I had initially followed the tutorial at https://docs.opnsense.org/manual/how-tos/wireguard-client-mullvad.html

Last time I used it 2-3 days ago it worked without issue. Yesterday the wireguard connection did not work anymore. I tried changing server but It did not correct the problem. The server was up and running correctly and I could test with my phone. My opnsense version is 23.7.4.
I notice that trying to access the system/firmware/status would stall indefinitely, as if the firewall itself was trying to reach internet through the now   not working wireguard connection. I notice that my wireguard connection would create a route 0.0.0.0 and prevent the firewall from accessing internet and listing status/updating. disabling the route creation in the wireguard page would prevent that from happening. The goal here was not that the firewall itself use the wireguard connection, it was just following the tutorial as instructed to allow other clients, through nat and sock5 to reach the vpn endpoint.

So far I have been unable to fix the issue preventing wireguard from working. The netstat and diagnostics show that we can send packages, but no reply are ever received. 

-- ho I see opn69 have a fix in a different thread, I'm going to look it up. I too have some firewalling around the vpn. I hope it's not going to be too complicated, I had a hard time figure it out.

Quote from: newsense on September 16, 2023, 02:32:40 PM
Check server status. If you touched our previously working configuration it's better to start over. There are no issues in 23.7.4 to be worried about here.

https://mullvad.net/en/servers
When you change how the software works and break working configurations along the way, I would call this an issue.

Quote from: opn69a on September 16, 2023, 07:07:43 PM
I use Mullvad as well and am assuming you're running into the same issue as me. If so... It's actually the upgrade to 23.7.3 that killed it, but OPNsense didn't reboot during that version like it did for 23.7.4, so it was kind of broken without knowing.

I got my VPN itself working via my latest post here: https://forum.opnsense.org/index.php?topic=35972.0
However, I'm still running into the issue where the firewall itself is not being routed through the VPN. This is likely due to the route that I deleted, mentioned at the end, but I'm not sure. I've got a separate thread here, where I'm trying to figure this out still: https://forum.opnsense.org/index.php?topic=35977.0

Note that the solution to the first link is probably mostly a hacky-version of it. TL;DR to it: I followed the Mullvad instructions from the pfsense+wireguard, but also checked the "disable routes" checkbox in the wireguard server settings (So it'd work on reboot).

Basically, it appears that when wireguard got moved from a software module over to the kernel, it took the changes for gateway configuration OR the routing table with it, resulting in it not working (i.e. killswitch was being hit so VPN "didn't work")... That's my current theory anyway. Perhaps the `0.0.0.0/0` allowed IPs is not a solution anymore for opnsense? No idea, lol. Just a bunch of messy theories I've got going in my head at the moment.
Thank you. I saw your thread earlier and after reading the pfsense guide I got it working as well. First I needed to manually add a gateway for IPv4 as instructed for pfsense (IP for the GW is 10.64.0.1, which is the IP of Mullvads IPv4 proxy) and then modified the "allow LAN to anywhere"-rule to use the gateway. It works fine now, I think I will be able to make it work from here. As a helpful note, if you are looking to make IPv6 work as well you can get the IPv6 address for the IPv6 gateway with the following snipped.
curl https://ipv6.am.i.mullvad.net --socks5-hostname 10.64.0.1

Quote from: n-dolce on September 17, 2023, 06:38:35 PM
Thank you. I saw your thread earlier and after reading the pfsense guide I got it working as well. First I needed to manually add a gateway for IPv4 as instructed for pfsense (IP for the GW is 10.64.0.1, which is the IP of Mullvads IPv4 proxy) and then modified the "allow LAN to anywhere"-rule to use the gateway. It works fine now, I think I will be able to make it work from here. As a helpful note, if you are looking to make IPv6 work as well you can get the IPv6 address for the IPv6 gateway with the following snipped.
curl https://ipv6.am.i.mullvad.net --socks5-hostname 10.64.0.1

Just out of curiosity, did you have to also do the "Disable Routes" checkbox in the VPN settings under the Wireguard tab, and assign your own gateway IP (10.64.0.1 or w/e)? This seems to be where I have some complications going on with my setup overall. Yes I have internet, and the network routes through the VPN, but I still have the issue of my firewall itself not being routed properly since that route of 0.0.0.0/1 gets deleted with checking that box to get the network to route properly. Just wanted to see if you had to do that, and if not, I would love to see if we can share notes to get that working. I even went as far as backing up my config and doing a "fresh install", but that still didn't get it to work properly. However, if your firewall still goes through wan and exposes a real IP from itself (ssh into it and do something like curling zx2c4.com/ip), then I guess we're in the same spot for now :)

I too lost connection to Mullvad through Wireguard after updating to 23.7.3 ( only after reboot ). But switching to the wireguard-go implementation instead of the kernel one, brought the connection back, without any changes to the configuration. If I switch back to the kernel implementation it will work until I reboot. So it seems something must have changed in the latest versions regarding the os-wireguard plugin compared to before, because as far as I recall nothing in my WG config has.

I had thought about doing that, but just be aware that wireguard-go is being deprecated, so as a result, you won't have the latest security fixes if you don't update to the kernel implementation. That's the only reason I didn't rollback to fix everything, although it definitely would have been a "quick fix" to the solution.

Quote from: n-dolce on September 17, 2023, 06:38:35 PM
When you change how the software works and break working configurations along the way, I would call this an issue.

When you don't change how the software works you don't get improvements bugfixes and security advisories shipped.

When you don't configure the software as it should it can break at any time when inconsistencies are fixed.

When you assume documentation is "perfect" but don't help make it perfect that's just what it is... not perfect.


Cheers,
Franco