OPNsense Forum
English Forums => Virtual private networks => Topic started by: Nikotine on September 12, 2021, 05:00:12 pm
-
I'm following this guide (https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html) to have only a few local hosts use a Mullvad remote Wireguard peer.
At the end of that guide, there are a few options to solve DNS leaks. I've chosen option 2, using a port forward for DNS requests to the Mullvad DNS server.
The problem is that now Chrome is complaining about the connection not being private (NET::ERR_CERT_COMMON_NAME_INVALID) or Opnsense complaining about a DNS rebind attack...
EDIT: I'm sorry for the huge screenshots. They looked fine on my screen, but huge after upload to imgur and inserting them here...
Wireguard connection to server in Switzerland:
(https://i.imgur.com/YywxP7z.jpg)
Wireguard gateway:
(https://i.imgur.com/Db0vvfq.jpg)
Firewall aliases (all_local_clients and local_hosts_remote_Mullvad).
I'm 10.25.9.10, the computer I'm typing this on.
Currently that alias is disabled, otherwise I can't reach this forum.
(https://i.imgur.com/Mqsv0yk.jpg)
Firewall rules.
Second rule is to have all traffic from local_hosts_remote_Mullvad alias, not destined to local networks, to use the Mullvad gateway.
(https://i.imgur.com/wxiy4Gb.jpg)
NAT outbound rule:
(https://i.imgur.com/qSGPGO9.jpg)
Then finally, to resolve the DNS leaks, NAT port forward of DNS request from hosts in local_hosts_remote_Mullvad alias to Mullvad's DNS server:
(https://i.imgur.com/jPSyRYM.jpg)
Things I've tried:
1. enabling Reflection for port forwards, Reflection for 1:1 and Automatic outbound NAT for Reflection. Didn't help.
(https://i.imgur.com/jQKVPft.jpg)
2. enabling, disabling or using system default NAT reflection setting in the DNS port forward rule. No difference.
(https://i.imgur.com/kWp02wn.jpg)
-
I'm just realizing that the error I get in Chrome is about the Let's Encrypt certificate I'm using locally, but it's only when I route the traffic over Wireguard...
-
What DNS servers do the relevant clients use otherwise?
-
Cloudflare 1.1.1.1 and 1.1.0.0.
-
So the port forward rule destination shouldn’t be “this firewall”, but instead the Cloudflare IPs or even “any”? That is, what you are trying to capture is the DNS request packets from the clients - if they are directly using Cloudflare rather than eg OPNsense (which is in turn forwarding to Cloudflare), then that is what the rule should be directed at.
-
Yes sorry, the clients use the firewall (unbound) as DNS server, who has DNS forwarding on, to Cloudflare.
But you're right, there's something wrong with this rule. Even if it would work, I wouldn't be able anymore to resolve local hostnames. I believe I should try to catch the DNS requests that are forwarded by unbound on the firewall to Cloudflare, to Mullvad instead, IF that request came from a client using Mullvad in the first place...
Not sure how to do that though. Packet tagging perhaps?
This was all very easy to do on Openwrt with the VPN policy routing plugin. It's a bit of a challenge in opnsense ;)
-
Yeah, that sounds like some more work
-
So... it seems the problem has nothing to do with DNS, I have disable all DNS forwarding stuff for the moment.
The problem is a certificate error. Chrome tries to use my local Let's Encrypt certificate whenever I'm trying to access a website, mullvad.net in this case...
(https://i.imgur.com/k4DINoT.jpg)
This blows my mind, any tips?
-
Ok, I disabled https for the GUI for a moment, to get that variable out of the loop.
Then I started googling and found that you, Greelan, are the author of the guide :)
I then found your link to this Imgur post: https://imgur.com/gallery/JBf2RF6.
Going through the screenshots in there, I found a mistake in my setup: when setting up the gateway, I hadn't used the fictive gateway IP from the local peer setup in Wireguard, but the Wireguard local tunnel address, entirely my mistake.
After having changed that, the Gateway went offline though, so I changed the monitor IP to 1.1.1.1 instead of the endpoint address.
Not sure why the latter isn't working.
(https://i.imgur.com/xiOsWds.jpg)
My selected hosts are now using the Wireguard connection!
I've also re-enabled https for GUI and that didn't break it :)
Now let's see how to get any DNS leaks fixed.
-
Good you found the problem.
For the monitor IP for Mullvad, do a traceroute while connected to the VPN. The first hop after OPNsense will be the tunnel IP at Mullvad’s end, that you can use as the monitor IP
-
Good tip! I changed the monitor IP.
-
With selective VPN routing now working, Mullvad keeps complaining about DNS leaks:
(https://i.imgur.com/iKPqATA.jpg)
I'm trying to implement option 2 of "dealing with DNS leaks" of the guide (https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html#dealing-with-dns-leaks).
All traffic coming from the hosts I want to use the Mullvad peer, is tagged with the first firewall rule.
I had to disable quick (red circle), otherwise it wouldn't process further rules.
The original rule to route traffic to the Wireguard gateway has been adjusted to match that tag.
(https://i.imgur.com/XjKgKx6.jpg?1)
This works, traffic for my selected hosts is still tunnelled via the Wireguard peer.
DNS leak exists.
Then I have added a port forward, to redirect DNS request, with the same packet tag, to Mullvad's DNS.
(https://i.imgur.com/OCRtylT.jpg)
But the DNS leak doesn't go away.
Unfortunately you can't setup a gateway in a port forward rule.
So am I correct that traffic will be tagged by my first firewall rule, then be port forwarded if the tagged traffic is DNS traffic, and then routed to the correct gateway by the second firewall rule?
-
I reverted back to the one firewall rule and removed the packet tagging.
Then I amended the port forward to forward ALL DNS traffic from the Mullvad clients to Mullvad's DNS server.
That works, I get all greens on Mullvad's check page, but local hostnames don't get resolved anymore obviously, which is only a small price to pay.
I suspect that when Unbound forwards a DNS request, the packet tag is removed.
-
Not sure that first rule will do anything. Being non-quick, it will only apply if the next rule doesn’t.
Also not sure if the port forward will achieve anything given the relevant clients are using the firewall as their DNS server, not Cloudflare.
What would work is simply a port forward that says any DNS requests coming into the firewall from the relevant clients is forwarded to Mullvad DNS.
However this will break local DNS resolution.
The solution you are really after is to only change the destination on egress - so if a DNS request is resolved locally, there is nothing to do, and only if OPNsense forwards it to Cloudflare is the destination then changed. But at that point the request is coming from the firewall, not the clients, and so is not using the tunnel.
Edit: snap!
-
We were cross-posting :)
My findings are the same... Thanks for the brainstorm anyway!
About this:
Not sure that first rule will do anything. Being non-quick, it will only apply if the next rule doesn’t.
It was working, that rule added a local tag to packets, which was then picked up by the second rule to route that traffic to the gateway.
Just the NAT port forward didn't seem to pick up that tagged traffic, probably because the DNS traffic I tried to capture now came from the firewall and the tag had been removed by Unbound.
-
Interesting observation. Not the behaviour I would have expected. And anyway, couldn’t the tag just be set on the main fw rule?
All this makes me wonder if unbound can handle split DNS. I have a Linux container that does that - any DNS request or reverse DNS request for local domains or IPs uses the main interface and local DNS servers. Everything else is sent via an OpenVPN tunnel interface to public DNS servers. I am not sure it is possible, but if unbound could do the same then that would address your issues. Or alternatively if unbound can be configured to use a different gateway and upstream DNS servers depending on the IP of the client making the request, that would be a potential solution. Just not sure what is achievable.
If local DNS resolution is important to you on the clients using the WG tunnel, you could implement a hack (assuming you are using a real domain for your local network) by configuring public DNS entries at your domain host that resolve to local IPs. Not best practice, and you would need to disable DNS rebinding protection on OPNsense, but it would achieve the outcome.
-
Interesting observation. Not the behaviour I would have expected. And anyway, couldn’t the tag just be set on the main fw rule?
The main fw rule excludes packets to RFC1918 networks. I needed all packets to be tagged.
Or alternatively if unbound can be configured to use a different gateway and upstream DNS servers depending on the IP of the client making the request, that would be a potential solution. Just not sure what is achievable.
Yeah, that's exactly what I was trying to achieve.
I don't think that's possible.
System > Settings > General let's you set up several DNS servers with different gateways, but it's not based on where the request is coming from.
If local DNS resolution is important to you on the clients using the WG tunnel, you could implement a hack (assuming you are using a real domain for your local network) by configuring public DNS entries at your domain host that resolve to local IPs. Not best practice, and you would need to disable DNS rebinding protection on OPNsense, but it would achieve the outcome.
Myeah, and then setup subdomains for all the different local hostnames. Seems not such a good practice indeed :)
In the end I can live without local DNS resolution, although I'm still sure there must be a trick I haven't thought of.
-
In the end I can live without local DNS resolution, although I'm still sure there must be a trick I haven't thought of.
Well, you could configure the hosts file on each client, but that’s obviously a bit of manual work depending on how many clients and how many internal DNS entries
-
Has anyone got ipv4 and ipv6 working at the same time? I can't specify two gateways for the local endpoint :'(
-
Good question. I asked the same a while back but haven’t received any feedback: https://github.com/opnsense/core/issues/5066.
However, I did manage to figure out how to create an IPv6 gateway, as indicated in that thread. I took the IPv6 tunnel IP given to me by my VPN provider (Mullvad), and included that as an additional Tunnel Address in the Local config. But rather than a /128, I specified it as a /127.
Then I allocated another address within that /127 as the gateway address, which worked.
I just didn’t get around to testing whether, once appropriate firewall rules and outbound NAT rules were created (like the equivalent IPv4 rules, but using the IPv6 gateway in the firewall rules), that would be enough. It is possible that, because Disable Routes is selected in the WG Local config, policy based routing established by the firewall rules is enough to get it working.
If I get some time, I will try to complete the config and testing. If you manage to try it out, let me know the results!
-
It works ;D
(https://i.imgur.com/fyeDsWT.png)
-
Ha, good to know! Perhaps there is no need to specify a gateway in the Local config at all…
I think I will update the how-to so that it has the IPv6 details.
-
@x390, I did my own testing and agree it seems to work.
One odd thing though is that IPv6 traceroutes from a client using the tunnel times out. Is that your experience? Ping works, browsing works, just not traceroute.
I use traceroute to find out the tunnel IP at Mullvad so I can use that as the monitor IP for the gateway. Works fine for IPv4.
BTW, OPNsense didn’t like it if I removed the IPv4 gateway from the Local config.
-
@Greelan, I tried running traceroute on opnsense here are the results.
# /usr/sbin/traceroute6 -w 2 -m '18' -s 'fc00:bbbb:bbbb:bb01::2:7bd8' '2606:4700:4700::1111'
traceroute6 to 2606:4700:4700::1111 (2606:4700:4700::1111) from fc00:bbbb:bbbb:bb01::2:7bd8, 18 hops max, 20 byte packets
1 fc00:bbbb:bbbb:bb01::1 20.729 ms 25.648 ms 21.192 ms
2 2400:fa80:1:11::1 23.273 ms 20.705 ms 22.750 ms
3 d5c:89d1:bc60:3::b 21.325 ms 25.349 ms 20.294 ms
4 d5c:89d1:bc60:3::a 21.198 ms 25.220 ms 20.966 ms
5 2402:1b80:1:11::36 20.337 ms 20.226 ms 23.783 ms
6 2402:1b80:1:11::26 25.638 ms 25.354 ms 21.307 ms
7 13335.syd.equinix.com 22.037 ms 40.306 ms 24.235 ms
8 2400:cb00:26:1024::6ca2:f850 23.891 ms
2400:cb00:26:1024::6ca2:f892 26.773 ms
2400:cb00:26:1024::6ca2:f836 25.054 ms
I had an issue where no matter what interface I choose to traceroute it would still go through the vpn. I solved this by changing the monitor ip in the gateways to monitor mullvads gateways instead.
I still need to do further testing.
-
Thanks. I tried it just now from OPNsense and that gave me the Mullvad tunnel IP (same as yours). So now I have my monitor IP. :)
Did you test though on a client behind OPNsense that is using the tunnel?
BTW, the behaviour you noticed of the traceroute always going via the tunnel may have been because you had Cloudflare as your monitor IP? OPNsense creates a static route for the monitor IP which means traffic to that IP will always be sent down the tunnel.
-
Hello.
I'm sorry if this thread hijack would seem unproper.
My issue with selective routing is accessing a specific public ip range (213.13.24.0/24) from an Openwrt Site "B" connected site-to-site through an Opnsense Site "A".
Configuring that subnet range on the Site "B" as "allowed ips" to the tunnel, so that Site "B" could access it through the Site "A", it isn't working as expected:
tracert 213.13.24.11
Tracing route to 213.13.24.11 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms OpenWRT.lan [192.168.0.1]
2 17 ms 14 ms 15 ms 10.0.0.1
3 * * * Request timed out.
4 * * * Request timed out.
5 * * * Request timed out.
6 * * * Request timed out.
7 * * * Request timed out.
8 * * * Request timed out.
9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
The site "B" LAN range is 192.168.0.0/24 with tunnel IP 10.0.0.2/32, the Site "A" is 192.168.10.0/24 with tunnel IP 10.0.0.1/32, and the WG tunnel range is 10.0.0.0/24. Both sites are connected to the internet with public IP addresses on their WAN interfaces.
The opnsense configuration is presented within the attachments bellow.
I'm hoping that someone could shed some light into this. :-)
Thanks.
-
@x390, nvm, it seems the tool I was using for IPv6 traceroute was broken, as it didn’t work regardless of whether the client was using the tunnel or not. I will test another tool later but expect it will be OK (Edit: Yes it is OK)
-
@Greelan, I've been trying to fix my setup and have got everything working except for port forwarding.
I have tried using nmap externally to see what is wrong and it shows that it is being filtered, but live view shows that it is being passed and redirected to the correct IP.
I have checked if the opnsense can connect with port probe and it is able to connect without issue.
Are you able to forward ports without issue?
-
Haven’t tried, but maybe you are hitting the reply-to issue for which there is a solution here (https://github.com/opnsense/core/issues/4389#issuecomment-865349224)
-
That did the trick! 🎉
Thanks Greelan for your help, I appreciated your support. 🙂