Messages

+1 to this please. I had a picture on my dashboard of some family that passed away, and it was always nice to go to my dashboard and see the image... Would really love to see this come back please.

What does your config look like?  Did you change any other settings besides setting up WG?

Also, I want to clarify something.  Are you attempting to initiate traffic from the OPNSense side to the client and that's not what's working?  Or are you initiating traffic from the client side to OPNSense?

What do your WG configs look like?  Are you using IPs, domains, etc?

Apologies, forgot to reach back out on this!

1. Made changes to nothing else except WG because I wanted to confirm it was WG and not something else
2. Neither work prior to restarting the service, both work after
3. IPs, following the guide word-for-word from the OPNsense wiki as well. Basically kept DNS out of this as much as possible.

Hoping Kinerg's bug report and next update will resolve all this though, so guess we'll see soon-ish? :D

I did a fresh installation of OPNsense on a separate machine to continue my troubleshooting with the issues I've been having with the new Wireguard kernel update to 2.0 and it looks like the 'main issue' is starting to lead to here: Wireguard doesn't establish a connection at boot.

I disabled Unbound DNS to rule that out as the culprit for the issues I've been having since updating to 23.7.3 (when Wireguard was updated to 2.0) and it looks like it wasn't related.

I backed up all my configuration _before_ doing the Wireguard setup. Can reboot, use internet, etc. nothing wrong whatsoever. But then the problem started after I setup Wireguard. Just for clariy, here's what I did:

Setup an endpoint, setup the local instance, and enable it. You get a Wireguard connection from the Firewall itself and can see this with the `wg` command. You can ping and curl and verify VPN access from the firewall. Before doing anything else - Reboot the firewall. Now go back to the Wireguard plugin, and you'll see it tries to send data, but never receives. It's completely stuck, even with the default NAT rules. Verified in an SSH session too, just to make sure. Go to the plugin, uncheck it, save, check it again, save... VPN connection is back and running. So I'm starting to think the main issue/bug I've been running into the past month or whatnot is potentially a result of this issue.

All in all, can +1 the report of WG not working unless manually restarting the plugin.

What do you mean by not being part of a major update?  OPNSense uses the YY.MM.minor_patch version format.  Updating from 23.1 to 23.7 is not a minor update and was actually the biggest update change I've seen from the project so far.  There have been a lot of information and posts about the changes and how it would affect things and not be as simple as prior updates.

Nah not 23.7 in general, that didn't cause issues for me. It was the update to 23.7.3 that broke everything. The update related to the wireguard module there is what appears to have broken things (when it went to v2.0).

It's not necessarily a concern of what I'm protecting against personally, otherwise I would have reverted to the previous wireguard version until this was resolved. It's more of an issue of it not working the way it was before with how traffic is routed, which is what resulted in my Internet outage for an entire day last week. And from what it looks like in the forum, many others had issues with their Wireguard VPN connectivity as well.

Also, my threat model is different than others. There may be those who have a very high threat model and their real IP being used even for updates or NTP instead of routed through their VPN is an issue. The goal of many with using a VPN as a client at OPNsense is to mask their entire network completely behind it, whether it's because of a particular threat model or just paranoia, that's their call - it just feels like this is a miss that was overlooked when implementing the new system.

That being said, as mentioned earlier, NTP is not the only thing being missed by the VPN routing. Anything done at the firewall itself is not routed, only the traffic being forwarded through it. Downloading DNS blocklists from external sources? Real IP. Grabbing the latest GeoIP lists? Real IP. Doing a ping? Real IP. Doing a DNS lookup that's not pointing to your own VPN's private IP? Real IP. It's a much larger issue than just NTP or Updates.

Had this been a part of a "major" update, it'd be understandable that things break. But when it's a minor update that's automatically pushed without warning of things breaking, and how the very core of how the VPN worked previously, it's kind of a big deal IMO.

With properly configured (V)Lan rules the only traffic the FW will do over the WAN link is as follows:


- WAN configuration, IPv4/IPv6

- bogons/opnsense updates -- which are all signed

- NTP traffic - which can be secured with Chrony


Even if your scope would be possible - which I highly doubt since it would create a loop - none of the three outlined activities above would be justified for VPN only.

I can understand WAN configuration (getting that Internet connection in the first place I assume) cannot go through the VPN, but I'm not sure I understand why the other two cannot be VPN-only.

For example, my personal laptop is running linux with the wireguard kernel implementation. Making the connection to the wireguard VPN tunnel of course uses my normal internet connection, but once that link has been established, everything else is automatically routed through the VPN. This includes NTP traffic and updates to the laptop. Why can OPNsense not be implemented in this way with the wireguard kernel update?

Also, what rules would need to be in place to try and get "other" traffic (i.e. curl commands or pings) to have the firewall be routed through the VPN connection?

Thanks

I had thought about doing that, but just be aware that wireguard-go is being deprecated, so as a result, you won't have the latest security fixes if you don't update to the kernel implementation. That's the only reason I didn't rollback to fix everything, although it definitely would have been a "quick fix" to the solution.

Latest updates -> Still trying to figure out a fix for this. I reached out to Mullvad, but it appears they're not going to help support this. Their response was:

We don't have any guide / official support for OPNsense neither do we do any testing on it.
You can read this:
https://docs.opnsense.org/manual/how-tos/wireguard-client-mullvad.html
Best regards,

Here's my latest email to them, so hoping anyone else here might be able to help get the OPNsense firewall to route itself through the VPN tunnel instead of my real IP:

I got it setup now to where the whole network still can connect to the internet, but the firewall itself still goes through my real IP - but it has internet now. I wanted to share the routes comparison of "disable routes" being checked, vs when it's not.

When leaving "Disable routes" unchecked (default), my connectivity breaks. Here are the routes that get generated from this:
default            [GATEWAY IP]       UGS        igb0
0.0.0.0/1          link#14            US          wg1
[PRIV_VPN_IP]      link#14            UH          lo0
[REAL_HOMEIP]      link#1             UHS         lo0
localhost          link#6             UH          lo0
128.0.0.0/1        link#14            US          wg1
192.168.0.0/28     link#9             U       bridge0
[FW_HOSTNAME]      link#9             UHS         lo0

where "GATEWAY IP" is my ISP's gateway ip address. When checking "Disable Routes" and setting the Gateway IP to "10.64.0.1", I get this:
default            [GATEWAY IP]       UGS        igb0
10.64.0.1          link#14            UHS         wg1
[PRIV_VPN_IP]      link#14            UH          lo0
[REAL_HOMEIP]      link#1             UHS         lo0
[REAL_HOMESUB]/24  link#1             U          igb0
localhost          link#6             UH          lo0
192.168.0.0/28     link#9             U       bridge0
[FW_HOSTNAME]      link#9             UHS         lo0

The diff on these is (- not working, + working):
- 0.0.0.0/1          link#14            US          wg1
- 128.0.0.0/1        link#14            US          wg1
+ 10.64.0.1          link#14            UHS         wg1
+  [REAL_HOMESUB]/24  link#1             U          igb0

Note, that I _cannot_ use 100.64.0.3 as the DNS server... I *have* to do 10.64.0.1 - I'm guessing this is because my firewall is not going through the VPN tunnel? Not entirely sure, but I've basically minimized my settings to the above. I also now have the following 2 firewall rules on the LAN interface:
Direction: in   |   Protocol: any   |   Source: LAN Net   |   Destination: LAN Net over any  |   Gateway: default
Direction: in   |   Protocol: any   |   Source: LAN Net   |   Destination: any over any  |   Gateway: Mullvad

(i.e. no rule specific to DNS).

Is anyone else able to confirm that they can get their Firewall to route through the VPN when it's connected to a Wireguard VPN? I really feel this should not be independent to Mullvad, but rather all Wireguard VPNs that have NAT setup to work. I already followed the guide a few days ago that Mullvad provided, but that didn't lead me to a solution.

Thank you. I saw your thread earlier and after reading the pfsense guide I got it working as well. First I needed to manually add a gateway for IPv4 as instructed for pfsense (IP for the GW is 10.64.0.1, which is the IP of Mullvads IPv4 proxy) and then modified the "allow LAN to anywhere"-rule to use the gateway. It works fine now, I think I will be able to make it work from here. As a helpful note, if you are looking to make IPv6 work as well you can get the IPv6 address for the IPv6 gateway with the following snipped.

Code Select Expand

curl https://ipv6.am.i.mullvad.net --socks5-hostname 10.64.0.1

Just out of curiosity, did you have to also do the "Disable Routes" checkbox in the VPN settings under the Wireguard tab, and assign your own gateway IP (10.64.0.1 or w/e)? This seems to be where I have some complications going on with my setup overall. Yes I have internet, and the network routes through the VPN, but I still have the issue of my firewall itself not being routed properly since that route of 0.0.0.0/1 gets deleted with checking that box to get the network to route properly. Just wanted to see if you had to do that, and if not, I would love to see if we can share notes to get that working. I even went as far as backing up my config and doing a "fresh install", but that still didn't get it to work properly. However, if your firewall still goes through wan and exposes a real IP from itself (ssh into it and do something like curling zx2c4.com/ip), then I guess we're in the same spot for now :)

be sure to update your firewall rules that have "outbound" traffic as your Wireguard's Gateway

Care to explain with a little more detail? I also have the issue, but i've already got an out firewall rule, and masquarading as the source of vpn in my setup. Thank you.

So I use Mullvad, and got the idea on doing this from their guide for pfsense+wireguard: https://mullvad.net/en/help/pfsense-with-wireguard/

The LAN interface rule that says "Default allow LAN to any" - Make sure to go into the edit menu on that and set the gateway to the VPN's gateway. This wasn't needed on the previous versions, likely due to the fact that the routing table worked with getting the firewall to automatically use the vpn as the default gateway for everything

Hope that helps.

I use Mullvad as well and am assuming you're running into the same issue as me. If so... It's actually the upgrade to 23.7.3 that killed it, but OPNsense didn't reboot during that version like it did for 23.7.4, so it was kind of broken without knowing.

I got my VPN itself working via my latest post here: https://forum.opnsense.org/index.php?topic=35972.0
However, I'm still running into the issue where the firewall itself is not being routed through the VPN. This is likely due to the route that I deleted, mentioned at the end, but I'm not sure. I've got a separate thread here, where I'm trying to figure this out still: https://forum.opnsense.org/index.php?topic=35977.0

Note that the solution to the first link is probably mostly a hacky-version of it. TL;DR to it: I followed the Mullvad instructions from the pfsense+wireguard, but also checked the "disable routes" checkbox in the wireguard server settings (So it'd work on reboot).

Basically, it appears that when wireguard got moved from a software module over to the kernel, it took the changes for gateway configuration OR the routing table with it, resulting in it not working (i.e. killswitch was being hit so VPN "didn't work")... That's my current theory anyway. Perhaps the `0.0.0.0/0` allowed IPs is not a solution anymore for opnsense? No idea, lol. Just a bunch of messy theories I've got going in my head at the moment.

So I managed to get my whole network back up and running for Wireguard, minus one caveat... It appears that the firewall itself cannot connect to the Internet when trying to directly access external sites. For example, I can't get the latest DNS blocklists for Unbound, and can't ping ip addresses (whether they're external or internal to the VPN). I noticed that the traffic attempts to go through my main ISP true IP address instead of the wireguard, resulting in it just failing.

How can I get the firewall to see "itself" as the IP address of the main wireguard ip address instead of the ISP's? Or to ask in an easier way, how can I force the firewall's traffic (outside the regular connection to my ISP for connectivity) to be completely routed through the VPN?

Alright, I figured it out. Took like 7 hours, but got it.... I had to change all my firewall rules to manually select the Wireguard Gateway, as well as add a few for things like DNS to not use the updated Gateway settings. I don't understand how changing Wireguard to the kernel version would have caused this, but it did.

So anyone else who might run into issues where you can't use Wireguard VPNs anymore, and your firewall rules are as complicated as mine might be (including port forwarding and all that), be sure to update your firewall rules that have "outbound" traffic as your Wireguard's Gateway. If you use a local DNS server on your OPNsense, but also have local LAN-to-LAN traffic, be sure to set a rule for the DNS right above that to use your Wireguard's gateway. So the order would be DNS for WG Gateway first, and then LAN-to-LAN traffic using the default gateway.

Hope this doesn't change in future updates.... Gonna have to do a major cleanup on my rules one of these days now that they're kinda scrambled from trying to fix this thing. lol

EDIT:
There's a part 2 to this... I ran into the issue again after rebooting and restarting Wireguard. Spent another many hours to find out that you have to go to the VPN connection and check the box "Disable routes". Otherwise it creates this route, which just doesn't work:

`ipv4   0.0.0.0/1   link#10   US   NaN   1420   wg1      [YOUR VPN NAME]`

I can't seem to be able to connect to any of my tunnels. I thought it was the upgrade to 23.7.4 but looking at my logs, I had uptime the entire 24.7.3 so I never had to reboot to get the newest kernel wireguard.

EDIT: Updated to reflect findings

I get constant tx but only a one time 92 bytes rx.

Anyone else experiencing the same problem?

I can see in my live view when testing connections that the LAN connection is accepted and the NAT to translate to my wireguard is successful, but it stops there. There's no log showing WG->website.

What changes could changing to the kernel version result in the connection breaking? As in, where should I look?

My allow ips include 0.0.0.0/0 and my own custom VPN gateway with IP 192.168.0.7. Already tried to set that to dynamic and that made no difference. So kind of stuck now

Hi there,

Is there any way to get the interfaces 'duplex' from the API? I tried a few of the `api/diagnostics/interface` endpoints with no luck (or I just didn't see it) and trying to just hit `/status_interface.php` with the api key doesn't work.

If not, is there a way to setup the logs to report which duplex is actively used?

I had my one cat5e cable show "100baseTX" suddenly. Re-plugging it in gave me the correct "1000baseT" it should have been before. I wanted to add in my monitoring and alerts for if this happens again to know if it was just a loose cable or if I'm going to have to replace this cable (or reclamp) due to it going faulty.

Preferably if I can get it into the logs and just have my grafana instance alert me on it, that'd be better. But if not, then an API endpoint would be the next step. Any assistance would be appreciated :)