Wireguard vs. DNS resolution (racing each other at boot time)

[oc]

Hello All,

First post here. I've been using OPN for about six months now, coming from various enterprise firewalls, and love it. Thank you Devs and community!

My setup is pretty complex, with multiple gateways pointing to various wireguard tunnels, routes, VLANs, policy-based routing, inbound wireguard server for mobile devices, ZA, crowdsec - and all works, and (fingers crossed) so far I've been spared of issues when updating/patching it.

However, one recurring issue I've experienced is that when I reboot the firewall (typically after an update), the following sequence happens:

1. watching it coming up in the console, wireguard takes a very long time starting (minutes) - but eventually it does

2. upon finally starting, S2S wireguard links are working (there are networks in a couple of other cities going through this OPN), but client VPNs (ex. mobile devices that always go through the OPN for protection when not in the home network) are connecting but can't resolve (so they can ping both internal as well as Internet IPs using the OPN public IP, but they cannot forward lookup external names)

3. the client issue reported at #2 is resolved immediately by restarting the inbound wireguard 'server' service (and I can do that even from afar, initially connecting to the tunnel and then to the OPNs internal management IP). And once the service restarts, all's good until the next reboot (and in fact, during some reboots wireguard got lucky and started after DNS so all was OK - but it's impredictable).

I have researched this online, and it appears to be because wireguard attempts to start before DNS (Unbound, I guess) is fully up and running, which leaves wireguard in an "in-betweeen" state (on top of delaying the startup by a lot of time, which is also very inconvenient).

The solutions suggested online:

a) do not use dynamic IPs names in tunnel names (aliases). While having static IPs is always desirable for VPN tunnel gateways, it's not always possible, as some ISPs aren't offering them (or at least not a the same service tier). So this is not really an option for me.

b) script it so that the wireguard service startup is delayed by a while, thereby giving the DNS resolver enough time to properly come up. This then is also supposed to expedite wireguard startup times. And while this is a reasonable solution, I didn't want to do that on my end, but rather pick the Dev's brains as to a more permanent workaround (for instance, there could be a condition testing if external DNS resolution works and only move on to other services relying on DNS, such as wireguard, after it's confirmed that it does). Other scripts baked into OPNsense are great (such as "Renewing DNS for Wireguard on stale connections" and "Reset WAN interface every day at 3am PT"), so perhaps there can be something to resolve this issue as well.

It's quite likely that a lot of users have to rely on dynamic IP name resolution for external VPN gateways, and it's a pity to let this one unaddressed, considering that the 'watchdog' function works brilliantly.

Again - thank you Devs and to the community. You are truly inspiring.

[meyergru]

The "Renew DNS for Wireguard on stale connections" cron job had a problem with that specific DNS outage on startup, but it has been fixed recently, now taking care of it correctly.

This fix is in the CE version, but can manually be applied to the BE as well.

[oc]

Thank you, meyergru.

The "Renew DNS for Wireguard on stale connections" cron job actually works for me - I've had the opportunity to test it by forcing a dynamic IP change on the other end, and the tunnel restored quickly thereafter. And btw, I'm on the CE and up to date (well, 26.7.3_8).

My issue is the slow boot time, and the fact that the inbound WG server doesn't resolve until a service restart. As stated initially, S2S WG works well even before the restart, it's only affecting the client-inbound WG connections.

IMO, it doesn't appear to be related to the "Renew DNS for Wireguard on stale connections" cron job, but rather to the boot sequence itself.

[meyergru]

For S2S connections, the cron job helps. For C2S, there should be nothing keeping your Wireguard server from starting, because the peer does not have to have an endpoint, so there is no DNS to be resolved that could keep the instance from coming up?