Multiple wireguard endpoints with public /29. How to route.

[ropeguru]

Posted this on Reddit, but thought I would try here also.

I am working on a setup where I have a single internet connection coming into the WAN port, two wireguard endpoints, zerotier, and a LAN. The two wireguard connection interfaces each have a /30 assigned to them for routing and support a unique public /29 across them. So two /29's, one on each WG endpoint.

At this point, the LAN is up and running as needed. The ZeroTier is up and configured on a bridge along with a ZT vlan extended out to my network. I have an IP on the bridge from the ZT network and a host on the vlan with a ZT IP and everything is working as expected.

Now, the issue. I cannot seem to understand how to make the wireguard connections work here. I have previously done this setup using Mikrotik CHR and Vyos where I could create multiple vrf's and routing tables to separate the default routes and attach each wireguard interface and the wireguard vlans to their respective vrf's.

In the setup for wireguard, the wireguard interface has an IP from a /30 assigned for point to point routing from the ISP. I then assign an IP from the /29 to the wireguard vlan interface to be used by hosts on the vlan as their default gateway. The associated route table gets a static default route entry pointing to the far side IP of the /30. This works really well and is a very simple setup to manage.

Where I am running into an issue with the wireguard on opensense is that there isn't an option for multiple vrf's or route tables, so from what I am reading, it seems I have to do this through the firewall itself. I cannot seem to be able to figure out how to make this happen since this is a "routed" setup.

I have tried creating a firewall rule on the /29 associated vlan for packets going out the interface to use the associated wg interface as the gateway, but it just doesn't seem to work. Is there something I am missing in the concept of using the firewall rules in this case or is a setup like this just not doable i OPNSense?

Image linked below I tried to give a visual idea of the setup.

https://imgur.com/a/7cwukH2



So some progress...

I have routing working to where if I put up a stateful service, ie. http/https, I can get to them from the outside with now issues. Tcpdump shows traffic in and out via the wireguard interface.

However, I cannot seem to get ICMP to honor the paths. The echo request from the internet comes in and the host sends the echo reply. But the reply is always following the default route of the main internet connection instead of going back out the wireguard connection. Pings outbound from the host, work as expected across the wireguard connecion.

Edit: I forgot to mention that I am also seeing TCP reset packets going out the default route instead of the wireguard interface. So it seems that the default is used anytime there isn't an associated session?

So as a test, I setup bind9 on my test host and it sends all udp port 53 replies back via the default gateway instead of the wireguard interface. So it seems anything that is considered "sessionless" or has no associated session uses the default gateway.

[zan]

Have you tried the "reply-to" set to the wg's gateway in the associated firewall's rules?