Firewall itself not going through VPN

Started by opn69a, September 15, 2023, 07:03:30 PM

Previous topic - Next topic
So I managed to get my whole network back up and running for Wireguard, minus one caveat... It appears that the firewall itself cannot connect to the Internet when trying to directly access external sites. For example, I can't get the latest DNS blocklists for Unbound, and can't ping ip addresses (whether they're external or internal to the VPN). I noticed that the traffic attempts to go through my main ISP true IP address instead of the wireguard, resulting in it just failing.

How can I get the firewall to see "itself" as the IP address of the main wireguard ip address instead of the ISP's? Or to ask in an easier way, how can I force the firewall's traffic (outside the regular connection to my ISP for connectivity) to be completely routed through the VPN?

Latest updates -> Still trying to figure out a fix for this. I reached out to Mullvad, but it appears they're not going to help support this. Their response was:

QuoteWe don't have any guide / official support for OPNsense neither do we do any testing on it.
You can read this:
https://docs.opnsense.org/manual/how-tos/wireguard-client-mullvad.html
Best regards,

Here's my latest email to them, so hoping anyone else here might be able to help get the OPNsense firewall to route itself through the VPN tunnel instead of my real IP:

QuoteI got it setup now to where the whole network still can connect to the internet, but the firewall itself still goes through my real IP - but it has internet now. I wanted to share the routes comparison of "disable routes" being checked, vs when it's not.

When leaving "Disable routes" unchecked (default), my connectivity breaks. Here are the routes that get generated from this:
default            [GATEWAY IP]       UGS        igb0
0.0.0.0/1          link#14            US          wg1
[PRIV_VPN_IP]      link#14            UH          lo0
[REAL_HOMEIP]      link#1             UHS         lo0
localhost          link#6             UH          lo0
128.0.0.0/1        link#14            US          wg1
192.168.0.0/28     link#9             U       bridge0
[FW_HOSTNAME]      link#9             UHS         lo0

where "GATEWAY IP" is my ISP's gateway ip address. When checking "Disable Routes" and setting the Gateway IP to "10.64.0.1", I get this:
default            [GATEWAY IP]       UGS        igb0
10.64.0.1          link#14            UHS         wg1
[PRIV_VPN_IP]      link#14            UH          lo0
[REAL_HOMEIP]      link#1             UHS         lo0
[REAL_HOMESUB]/24  link#1             U          igb0
localhost          link#6             UH          lo0
192.168.0.0/28     link#9             U       bridge0
[FW_HOSTNAME]      link#9             UHS         lo0

The diff on these is (- not working, + working):
- 0.0.0.0/1          link#14            US          wg1
- 128.0.0.0/1        link#14            US          wg1
+ 10.64.0.1          link#14            UHS         wg1
+  [REAL_HOMESUB]/24  link#1             U          igb0

Note, that I _cannot_ use 100.64.0.3 as the DNS server... I *have* to do 10.64.0.1 - I'm guessing this is because my firewall is not going through the VPN tunnel? Not entirely sure, but I've basically minimized my settings to the above. I also now have the following 2 firewall rules on the LAN interface:
Direction: in   |   Protocol: any   |   Source: LAN Net   |   Destination: LAN Net over any  |   Gateway: default
Direction: in   |   Protocol: any   |   Source: LAN Net   |   Destination: any over any  |   Gateway: Mullvad

(i.e. no rule specific to DNS).

Is anyone else able to confirm that they can get their Firewall to route through the VPN when it's connected to a Wireguard VPN? I really feel this should not be independent to Mullvad, but rather all Wireguard VPNs that have NAT setup to work. I already followed the guide a few days ago that Mullvad provided, but that didn't lead me to a solution.

With properly configured (V)Lan rules the only traffic the FW will do over the WAN link is as follows:


- WAN configuration, IPv4/IPv6

- bogons/opnsense updates -- which are all signed

- NTP traffic - which can be secured with Chrony


Even if your scope would be possible - which I highly doubt since it would create a loop - none of the three outlined activities above would be justified for VPN only.

Quote from: newsense on September 19, 2023, 09:50:17 PM
With properly configured (V)Lan rules the only traffic the FW will do over the WAN link is as follows:


- WAN configuration, IPv4/IPv6

- bogons/opnsense updates -- which are all signed

- NTP traffic - which can be secured with Chrony


Even if your scope would be possible - which I highly doubt since it would create a loop - none of the three outlined activities above would be justified for VPN only.

I can understand WAN configuration (getting that Internet connection in the first place I assume) cannot go through the VPN, but I'm not sure I understand why the other two cannot be VPN-only.

For example, my personal laptop is running linux with the wireguard kernel implementation. Making the connection to the wireguard VPN tunnel of course uses my normal internet connection, but once that link has been established, everything else is automatically routed through the VPN. This includes NTP traffic and updates to the laptop. Why can OPNsense not be implemented in this way with the wireguard kernel update?

Also, what rules would need to be in place to try and get "other" traffic (i.e. curl commands or pings) to have the firewall be routed through the VPN connection?

Thanks

A reason not to have NTP going through the VPN is that if your clock drifts too much you won't be able to connect to the VPN at all due to cert mismatch.

That said, I'm not sure why you want all of OPNSense to use the VPN.  Is there a particular concern that you're trying to protect against?

It's not necessarily a concern of what I'm protecting against personally, otherwise I would have reverted to the previous wireguard version until this was resolved. It's more of an issue of it not working the way it was before with how traffic is routed, which is what resulted in my Internet outage for an entire day last week. And from what it looks like in the forum, many others had issues with their Wireguard VPN connectivity as well.

Also, my threat model is different than others. There may be those who have a very high threat model and their real IP being used even for updates or NTP instead of routed through their VPN is an issue. The goal of many with using a VPN as a client at OPNsense is to mask their entire network completely behind it, whether it's because of a particular threat model or just paranoia, that's their call - it just feels like this is a miss that was overlooked when implementing the new system.

That being said, as mentioned earlier, NTP is not the only thing being missed by the VPN routing. Anything done at the firewall itself is not routed, only the traffic being forwarded through it. Downloading DNS blocklists from external sources? Real IP. Grabbing the latest GeoIP lists? Real IP. Doing a ping? Real IP. Doing a DNS lookup that's not pointing to your own VPN's private IP? Real IP. It's a much larger issue than just NTP or Updates.

Had this been a part of a "major" update, it'd be understandable that things break. But when it's a minor update that's automatically pushed without warning of things breaking, and how the very core of how the VPN worked previously, it's kind of a big deal IMO.

Quote from: opn69a on September 20, 2023, 04:29:06 PM
It's not necessarily a concern of what I'm protecting against personally, otherwise I would have reverted to the previous wireguard version until this was resolved. It's more of an issue of it not working the way it was before with how traffic is routed, which is what resulted in my Internet outage for an entire day last week. And from what it looks like in the forum, many others had issues with their Wireguard VPN connectivity as well.

Also, my threat model is different than others. There may be those who have a very high threat model and their real IP being used even for updates or NTP instead of routed through their VPN is an issue. The goal of many with using a VPN as a client at OPNsense is to mask their entire network completely behind it, whether it's because of a particular threat model or just paranoia, that's their call - it just feels like this is a miss that was overlooked when implementing the new system.

That being said, as mentioned earlier, NTP is not the only thing being missed by the VPN routing. Anything done at the firewall itself is not routed, only the traffic being forwarded through it. Downloading DNS blocklists from external sources? Real IP. Grabbing the latest GeoIP lists? Real IP. Doing a ping? Real IP. Doing a DNS lookup that's not pointing to your own VPN's private IP? Real IP. It's a much larger issue than just NTP or Updates.

Had this been a part of a "major" update, it'd be understandable that things break. But when it's a minor update that's automatically pushed without warning of things breaking, and how the very core of how the VPN worked previously, it's kind of a big deal IMO.

I never said NTP was the only thing missed.  I simply provided a reason why sending NTP through the VPN could cause a problem.

What do you mean by not being part of a major update?  OPNSense uses the YY.MM.minor_patch version format.  Updating from 23.1 to 23.7 is not a minor update and was actually the biggest update change I've seen from the project so far.  There have been a lot of information and posts about the changes and how it would affect things and not be as simple as prior updates.

Quote from: CJ on September 20, 2023, 05:30:12 PM
What do you mean by not being part of a major update?  OPNSense uses the YY.MM.minor_patch version format.  Updating from 23.1 to 23.7 is not a minor update and was actually the biggest update change I've seen from the project so far.  There have been a lot of information and posts about the changes and how it would affect things and not be as simple as prior updates.

Nah not 23.7 in general, that didn't cause issues for me. It was the update to 23.7.3 that broke everything. The update related to the wireguard module there is what appears to have broken things (when it went to v2.0).

Quote from: opn69a on September 20, 2023, 05:43:41 PM
Nah not 23.7 in general, that didn't cause issues for me. It was the update to 23.7.3 that broke everything. The update related to the wireguard module there is what appears to have broken things (when it went to v2.0).

That update was fixing the issues caused caused by the interaction between the new kernel and wireguard IIRC.  So while it may have been working for you, it was broken for a bunch of other people.

October 05, 2023, 03:19:04 AM #9 Last Edit: October 05, 2023, 03:21:21 AM by 36thchamber
Quote from: opn69a on September 20, 2023, 04:29:06 PM
Downloading DNS blocklists from external sources? Real IP. Grabbing the latest GeoIP lists? Real IP. Doing a ping? Real IP. Doing a DNS lookup that's not pointing to your own VPN's private IP? Real IP.
ACME updates? Real IP. DDNS updates? Real IP. Literally announcing your real IP when publishing DNS for your VPN IP. Any plugin connections? Real IP. Scary.

So rare to find anything related. This one has some solution tho it doesn't work, and all behaves in exactly opposite way (defaults, rule creation):
https://forum.opnsense.org/index.php?topic=23399.0

This has been my nr.1 priority for weeks and i still couldn't solve it. I want the firewall to act like any other device, it's just unacceptable to have a tons of code and plugins act like an invisible actor without filtering.

No idea how (except running 2nd instance of Opnsense which would be ridiculous). I only managed to shake out Unbound connections from WAN, and only thanks to static routes (praying the resolving IPs won't change in time).