Topics

I'm having a bit of trouble setting up two wireguard client connections, with two different WAN interfaces.

I have WAN1 and WAN2, two independent connections to the internet. WAN2 generally has higher bandwidth and is the preferred connection in my gateway group for WAN_FAILOVER.

I have two wireguard clients configured. WG_WAN1 and WG_WAN2. These connect to two separate endpoints. I want WG_WAN1 to only connect via WAN1 and WG_WAN2 to only connect via WAN2. So far I've achieved this by adding static routes to their endpoint IPs, defining which interface I want to route the traffic on.

Now normally this works great and everything functions as expected. The trouble I run into is when WAN2 goes down for any appreciable time and things failover to WAN1. Initially I see WG_WAN2 go down as expected, but if WAN2 stays down for a while, eventually WG_WAN2 will come back up, routed through WAN1. This is the part that I do not want to happen.

I do have default gateway switching turned on in the firewall, as I want traffic originated from it to handle a single WAN failure (for DNS). Everything else is policy routed through my gateway groups and works great. I believe that a static route should have precedence over discovered routes, but I may be wrong there.

I should also add that I'm running these wireguard clients with their own assigned interfaces, if that wasn't obvious from context.

Am i missing a crucial element in how to bind a WG client to a particular WAN interface in a failover setup?

Alright, let me preface this with the fact that I've had a wireguard tunnel up and working to NAT my traffic outbound for a long while now. All of it sudden it has become very inconsistent with no observable changes on my end. I'm on the latest version of OPNSense.

I have a fairly complex network by homelab standards. There are several VLANs on my parent LAN interface, as well as a few separate hardware interfaces, all talking to an old protectli vault running OPNSense. My WAN is a single interface feeding upstream to a separate cellular modem running OpenWRT. Unfortunately there's an extra NAT layer due to that, but that's a different discussion...
I have a wireguard client setup with Mullvad. The config has always worked great. The only thing I changed after setting it up initially was disabling route pulls and configuring a manual gateway, so it wouldn't shove itself in the firewall's routing table as a default route.
I have policy routes setup so that most of my traffic ends up NATing out the wireguard tunnel, with the exception of one entire VLAN, and a couple select hosts elsewhere that just NAT out the modem's gateway. Notably, the firewall itself does not NAT out the wireguard tunnel.

This has always worked until recently.

I'm now having problems where the wireguard gateway will jump to 100% packet loss seemingly randomly. The tunnel dies, and so does the connectivity for those policy routed hosts. If I deactivate and reactivate wireguard, either in the web GUI or with a `wg-quick down wg0 && wg-quick up wg0`, it never gets another handshake. I have to reboot the firewall entirely to get wireguard to come back. I have my endpoint config pointing to an IP address rather than a hostname, so I know it's not DNS. I can still talk to the internet and even that wireguard endpoint specifically from the firewall's CLI, so I know reaching it isn't a problem. Why does wireguard just die though?

A bit more background, although I don't believe it's too important. I did switch to the wireguard kernel module when it became available. I know that it was for testing and potentially unstable. It worked great for weeks, maybe even over a month. When these failures started happening, I uninstalled the kernel module and went back to wireguard-go, just to be in a "fully supported" state so nothing could be blamed on beta testing. The exact same thing happens.

I've gone through what feels like umpteen logs, and I cannot find anything related to the cause. I see the events logged in various places when the wireguard tunnel goes down, but I don't see anything right before that is causing it.

Now I am here, frustrated and desparately asking for the help of this community, as this is almost completely breaking the usability of my network at random intervals.

Update was successful but left me with some questions about the netmap features. I was using the 20.7.3-netmap kernel in the last release and it was working excellently. I have a 4-port NIC running the em driver. With the netmap kernel I had suricata running in IPS mode on my WAN and VPN client interfaces, no problems. I also had Sensei running on my parent LAN for a few VLANs in native netmap mode, no problems.

Maybe I'm interpretting your release notes wrong, but it sounded like the netmap kernel was rolled into the default for 20.7.4 and what I'm experiencing leads me to believe it's not. I've got Sensei working okay, although I can't comment on consistency yet. Whenever I enable Suricata though, it starts dropping packets like mad. I looked for a 20.7.4-netmap but don't see it. Am I missing something?