[SOLVED] Wireguard selective routing

[Greelan]

Interesting observation. Not the behaviour I would have expected. And anyway, couldn't the tag just be set on the main fw rule?

All this makes me wonder if unbound can handle split DNS. I have a Linux container that does that - any DNS request or reverse DNS request for local domains or IPs uses the main interface and local DNS servers. Everything else is sent via an OpenVPN tunnel interface to public DNS servers. I am not sure it is possible, but if unbound could do the same then that would address your issues. Or alternatively if unbound can be configured to use a different gateway and upstream DNS servers depending on the IP of the client making the request, that would be a potential solution. Just not sure what is achievable.

If local DNS resolution is important to you on the clients using the WG tunnel, you could implement a hack (assuming you are using a real domain for your local network) by configuring public DNS entries at your domain host that resolve to local IPs. Not best practice, and you would need to disable DNS rebinding protection on OPNsense, but it would achieve the outcome.

[Nikotine]

Interesting observation. Not the behaviour I would have expected. And anyway, couldn't the tag just be set on the main fw rule?

The main fw rule excludes packets to RFC1918 networks. I needed all packets to be tagged.

Or alternatively if unbound can be configured to use a different gateway and upstream DNS servers depending on the IP of the client making the request, that would be a potential solution. Just not sure what is achievable.

Yeah, that's exactly what I was trying to achieve.
I don't think that's possible.
System > Settings > General let's you set up several DNS servers with different gateways, but it's not based on where the request is coming from.

If local DNS resolution is important to you on the clients using the WG tunnel, you could implement a hack (assuming you are using a real domain for your local network) by configuring public DNS entries at your domain host that resolve to local IPs. Not best practice, and you would need to disable DNS rebinding protection on OPNsense, but it would achieve the outcome.

Myeah, and then setup subdomains for all the different local hostnames. Seems not such a good practice indeed :)
In the end I can live without local DNS resolution, although I'm still sure there must be a trick I haven't thought of.

[Greelan]

In the end I can live without local DNS resolution, although I'm still sure there must be a trick I haven't thought of.

Well, you could configure the hosts file on each client, but that's obviously a bit of manual work depending on how many clients and how many internal DNS entries

[x390]

Has anyone got ipv4 and ipv6 working at the same time? I can't specify two gateways for the local endpoint  :'(

[Greelan]

Good question. I asked the same a while back but haven't received any feedback: https://github.com/opnsense/core/issues/5066.

However, I did manage to figure out how to create an IPv6 gateway, as indicated in that thread. I took the IPv6 tunnel IP given to me by my VPN provider (Mullvad), and included that as an additional Tunnel Address in the Local config. But rather than a /128, I specified it as a /127.

Then I allocated another address within that /127 as the gateway address, which worked.

I just didn't get around to testing whether, once appropriate firewall rules and outbound NAT rules were created (like the equivalent IPv4 rules, but using the IPv6 gateway in the firewall rules), that would be enough. It is possible that, because Disable Routes is selected in the WG Local config, policy based routing established by the firewall rules is enough to get it working.

If I get some time, I will try to complete the config and testing. If you manage to try it out, let me know the results!

[x390]

It works ;D

[Greelan]

Ha, good to know! Perhaps there is no need to specify a gateway in the Local config at all...

I think I will update the how-to so that it has the IPv6 details.

[Greelan]

@x390, I did my own testing and agree it seems to work.

One odd thing though is that IPv6 traceroutes from a client using the tunnel times out. Is that your experience? Ping works, browsing works, just not traceroute.

I use traceroute to find out the tunnel IP at Mullvad so I can use that as the monitor IP for the gateway. Works fine for IPv4.

BTW, OPNsense didn't like it if I removed the IPv4 gateway from the Local config.

[x390]

@Greelan, I tried running traceroute on opnsense here are the results.

Code Select Expand

# /usr/sbin/traceroute6 -w 2 -m '18' -s 'fc00:bbbb:bbbb:bb01::2:7bd8'   '2606:4700:4700::1111'
traceroute6 to 2606:4700:4700::1111 (2606:4700:4700::1111) from fc00:bbbb:bbbb:bb01::2:7bd8, 18 hops max, 20 byte packets
1  fc00:bbbb:bbbb:bb01::1  20.729 ms  25.648 ms  21.192 ms
2  2400:fa80:1:11::1  23.273 ms  20.705 ms  22.750 ms
3  d5c:89d1:bc60:3::b  21.325 ms  25.349 ms  20.294 ms
4  d5c:89d1:bc60:3::a  21.198 ms  25.220 ms  20.966 ms
5  2402:1b80:1:11::36  20.337 ms  20.226 ms  23.783 ms
6  2402:1b80:1:11::26  25.638 ms  25.354 ms  21.307 ms
7  13335.syd.equinix.com  22.037 ms  40.306 ms  24.235 ms
8  2400:cb00:26:1024::6ca2:f850  23.891 ms
    2400:cb00:26:1024::6ca2:f892  26.773 ms
    2400:cb00:26:1024::6ca2:f836  25.054 ms

I had an issue where no matter what interface I choose to traceroute it would still go through the vpn. I solved this by changing the monitor ip in the gateways to monitor mullvads gateways instead.

I still need to do further testing.

[Greelan]

Thanks. I tried it just now from OPNsense and that gave me the Mullvad tunnel IP (same as yours). So now I have my monitor IP. :)

Did you test though on a client behind OPNsense that is using the tunnel?

BTW, the behaviour you noticed of the traceroute always going via the tunnel may have been because you had Cloudflare as your monitor IP? OPNsense creates a static route for the monitor IP which means traffic to that IP will always be sent down the tunnel.

[ReDaLeRt]

Hello.

I'm sorry if this thread hijack would seem unproper.

My issue with selective routing is accessing a specific public ip range (213.13.24.0/24) from an Openwrt Site "B" connected site-to-site through an Opnsense Site "A".

Configuring that subnet range on the Site "B" as "allowed ips" to the tunnel, so that Site "B" could access it through the Site "A", it isn't working as expected:

Code Select Expand

tracert 213.13.24.11

Tracing route to 213.13.24.11 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  OpenWRT.lan [192.168.0.1]
  2    17 ms    14 ms    15 ms  10.0.0.1
  3     *        *        *     Request timed out.
  4     *        *        *     Request timed out.
  5     *        *        *     Request timed out.
  6     *        *        *     Request timed out.
  7     *        *        *     Request timed out.
  8     *        *        *     Request timed out.
  9     *        *        *     Request timed out.
10     *        *        *     Request timed out.
11     *        *        *     Request timed out.
12     *        *        *     Request timed out.
13     *        *        *     Request timed out.
14     *        *        *     Request timed out.
15     *        *        *     Request timed out.

The site "B" LAN range is 192.168.0.0/24 with tunnel IP 10.0.0.2/32, the Site "A" is 192.168.10.0/24 with tunnel IP 10.0.0.1/32, and the WG tunnel range is 10.0.0.0/24. Both sites are connected to the internet with public IP addresses on their WAN interfaces.

The opnsense configuration is presented within the attachments bellow.

I'm hoping that someone could shed some light into this. :-)

Thanks.

[Greelan]

@x390, nvm, it seems the tool I was using for IPv6 traceroute was broken, as it didn't work regardless of whether the client was using the tunnel or not. I will test another tool later but expect it will be OK (Edit: Yes it is OK)

[x390]

@Greelan, I've been trying to fix my setup and have got everything working except for port forwarding.

I have tried using nmap externally to see what is wrong and it shows that it is being filtered, but live view shows that it is being passed and redirected to the correct IP.

I have checked if the opnsense can connect with port probe and it is able to connect without issue.

Are you able to forward ports without issue?

[Greelan]

Haven't tried, but maybe you are hitting the reply-to issue for which there is a solution here

[x390]

That did the trick! 🎉

Thanks Greelan for your help, I appreciated your support. 🙂