How can I automatically restart wg instance, if gw down?

[tessus]

It might sound like a VPN issue, but my workaround seems more like a question for this forum.

Short description: I am using 3 wireguard instances, one of which is for a VPN provider I use and I have also setup a GW which is used by specific clients in my network. All good. Works great.
However, today my network showed that this GW was offline (red status dot). My fix was to disable and then enable this wg2 instance. Aaaand, it became green again.

So here is my question: how do I automate the restart of the wg2 instance, if the gateway (of the interface assigned to wg2) goes down?

Before using Wireguard to connect to my VPN provider, I used OpenVPN and for some reason the openvpn connection and the gateway always recovered by itself. But the Wireguard GW does not recover on its own. Strange, because wireguard is actually great becoming automatically online again after issues with an endpoint.
I suspect it is rather complicated to find out why the gateway does not recover or why the Wireguard instance seems to be in a wonky state. (Should it happen again, I will collect as much info as possible. Maybe opening a topic in the VPN forum is then warranted.)

Anyhoo, retarting the wg2 instance is an easy fix, but it has to happen automatically. I just do not know how. Any ideas, pointers, black magic rituals?

[meyergru]

There is a cron job "Restart Wireguard on stale connections" that you can enable.

[tessus]

Thanks for the reply.

I checked the cron jobs before posting and the ones related to wg are:

- Renew DNS for Wireguard on stale connections
- Restart Wireguard

Both of which are not what I want. Restarting all my wireguard connections unconditionally is rather interrupting and I won't do that.
And renewing the DNS won't help, if wireguard is wonky in the first place.

But let's say there was a "Restart Wirguard instance on stale connections", how often would I run that? Every minute?
I need some other trigger. e.g. a hook to run a script when a gw goes down, or something like that. But then I still need to know how to restart a specififc wireguard instance on FreeBSD and/or OPNsense. I am great on Linux but rather inexperienced with FreeBSD.

[BrandyWine]

Is there an action for gateway fail in opnsense gui?
If so you can invoke a script to bounce wg2?

Using dpinger to Invoke a Script on Monitoring Failures
Overview of dpinger

dpinger is a daemon used in pfSense and OPNsense for monitoring gateway status. It sends ICMP echo requests (pings) to specified IP addresses to determine if a gateway is online or offline. If a gateway fails to respond, dpinger can trigger actions based on its configuration.
Setting Up Script Invocation

To invoke a script when monitoring fails, follow these steps:

    Create Your Script
        Write a shell script that performs the desired actions when a gateway goes down. Ensure it has executable permissions.
        Example script (/usr/local/bin/gateway_fail.sh):

bash

        #!/bin/sh
        echo "Gateway is down!" >> /var/log/gateway_monitor.log
        # Add additional commands here

    Configure dpinger
        Access the pfSense or OPNsense web interface.
        Navigate to System > Routing > Gateways.
        Edit the gateway you want to monitor.

    Set Up the Script Trigger
        In the gateway settings, look for the Advanced section.
        Find the option for "Execute command on gateway failure".
        Enter the path to your script (e.g., /usr/local/bin/gateway_fail.sh).

    Testing the Configuration
        Simulate a gateway failure by disconnecting the network or changing the monitored IP.
        Check the log file (/var/log/gateway_monitor.log) to confirm that the script executed successfully.

Additional Considerations

    Permissions: Ensure that the script has the correct permissions to execute and that the user running dpinger has permission to execute the script.

    Logging: Implement logging within your script to track its execution and any errors that may occur.

    Recovery Actions: You can also create a script for when the gateway comes back online by using the "Execute command on gateway recovery" option in the same settings.

By following these steps, you can effectively use dpinger to invoke a script when monitoring fails, allowing for automated responses to gateway issues.

[tessus]

Unfortunately there is no "Execute command on gateway failure" in my advanced section for gateways. I am using OPNsense 25.7.6-amd64.

These are the ones available after I slide the advanced button (image availabe for 7 days):

[tessus]

This would have been exactly what I was looking for. Well, and how to actually restart a wg2 instance on OPNsense via the command line. I certainly can't do a: systemctl restart wg-quick@wg2

[tessus]

@BrandyWine can you please let me know where I can find the option "Execute command on gateway failure"? It is not in the advanced gateway options. Are you using a patched version of OPNsense? If so, how do I get that patch? If not, how can you have that option, when I do not?

[tessus]

I am sorry to bump this again, but both answers don't work, because the mentioned options do not exist.

The situation I described isn't a far-fetched theoretical problem, but a real-life possible scenario that is actually quite common for a router/firewall: if gw is down, run a script or trigger an action.

[Patrick M. Hausen]

As far as I know "Renew DNS for Wireguard on stale connections" does exactly what you want, only the description might be not quite appropriate. In the majority of cases a WG connection goes "stale" because of dynamic IP addresses. So that's the primary use case.

"Renew DNS" means "restart that connection so it will do a fresh DNS lookup".

So you get your restart. Did you try that cron job, yet?

[DEC670airp414user]

yep the remote command for openvpn is certainly missed.   if one server would go down it would round robin to the next one on the list.
I have no idea if the following method works but this is what I have setup

for a vpn instance I've added multiple "peers" to it.    thinking if the first one goes down, it will connect to the next peer.    I have not had a wireguard instance go down recently to see if it works.  In theory you think it would.

see attached screen shot

[tessus]

"Renew DNS" means "restart that connection so it will do a fresh DNS lookup".

Ok, then it makes sense and I agree with you: the wording is a bit off. ;-)

So you get your restart. Did you try that cron job, yet?

Not yet, because I do not know how often I should run it (and my GW hasn't gone down since). Running the cron job every minute is probably a strain on the system. 15 minutes could leave my VPN dead for 14 minutes and 59 seconds in the worst case. While I agree with you that it is workaround and might solve the issue, it is not as useful as triggering a command/action if the gw goes down.
I read up on dpinger and it seems that commands can be triggered and run, but I suppose that functionality was not added to opnsense.
I will open a feature request.

for a vpn instance I've added multiple "peers" to it.    thinking if the first one goes down, it will connect to the next peer. 

I first thought so as well and I have even created another peer but left it disabled. Because according to the Wireguard architecture peers are always active when enabled, unless I misunderstood it. Afaik there is no setting in Wireguard to allow for fail over connections. This means both are active (which counts towards your concurrent connections) and I have no idea what this does to sessions. e.g. If you connect to a web site every packet could come from either one of two IP addresses (or how many peers you have setup). This will certainly raise a flag, if continous packets have alternate IP addresses in their TCP headers.

[Patrick M. Hausen]

Running the cron job every minute is probably a strain on the system.

IMHO a cron job once per minute that decides there is nothing to do should do no harm. A minute is an eternity in CPU time.

[tessus]

Running the cron job every minute is probably a strain on the system.

IMHO a cron job once per minute that decides there is nothing to do should do no harm. A minute is an eternity in CPU time.

Thanks, I will set it up.

"A minute is an eternity in CPU time." Totally agree, but I haven't checked how these cron actions are actually written. python, php, ...? the first 2 come with quite some overhead, since the interpreter has to be loaded every time. This means I could have a spike every minute. But I guess I'll see.

[meyergru]

The overhead is minimal (I wrote the job) - I call it every 5 minutes. Note that you only need it on the VPN side that initiates the connection to find the target if it changes IPs.