MutiWAN and Reset States

[mliebherr]

Hello,

we are using MultiWAN with 2 Uplinks with:
- Gateway switching (Allow default gateway switching => enabled)
- Kill States (  Disable State Killing on Gateway Failure  => not ticked)
- Sticky Connections ( Use sticky connections => not ticked)

On top of that i run a OpenVPN Client Connection (TCP)

When i produce the active Gateway failure, the Gateway swichting jumps in, the OpenVPN Tunnel times out and the takeover is fine. It also seems to do a TCP States Reset since my SSH Tunnel/Access dies.

HOWEVER: If i switch back on the Gateway the  Active Gateway switches back to the main one again, BUT the TCP States does not get killed.

The SSH Session is still active. Not states Reset seem to happen.
If i kill the ESTABLISHED connection in the "States Dump" GUI then it will start to connect via the active/correct gateway.

So wonder if:
 -if i set up something wrong?
- the state reset just happens by design on the 1st failover
- the state reset function is a bug and should be triggered when jumping back to the primary interface





Thanks,
Michael

[FullyBorked]

I feel this pain, I've experienced exactly this for the entire time I've used OPNsense and multi-wan.  Failing back to the primary connection feels hit and miss.  It did get a LOT better for me after disabling sticky connections.  But for me my backup is metered LTE and I really don't want connections staying on that link if the primary is available.  Sometimes after my primary comes back I have to reboot the LTE modem to force connections back over.  Everything will eventually fail back, but I can't find a way to force it.  I'm a Watchguard firewall guy and it has an option to force connections back, even though it can be briefly disruptive. Really wish that was an available option in OPNsense. 

[Simser]

I also have some problems with MultiWAN and connections sometimes don't want to let go of the metered, LTE failover WAN.

Do you also experience the problem with a gateway failover group, the internet (and VPN, etc) gets disrupted when the secondary (failover, tier 2) WAN gets down despite the primary (standard, tier 1) still working? I guess it is the state killing on gateway failure... but why does it kick in when the failover with no traffic other than monitoring goes down. This behavior doesn't seem ideal to me. So I'd like to hear what other people experienced.

[FullyBorked]

I also have some problems with MultiWAN and connections sometimes don't want to let go of the metered, LTE failover WAN.

Do you also experience the problem with a gateway failover group, the internet (and VPN, etc) gets disrupted when the secondary (failover, tier 2) WAN gets down despite the primary (standard, tier 1) still working? I guess it is the state killing on gateway failure... but why does it kick in when the failover with no traffic other than monitoring goes down. This behavior doesn't seem ideal to me. So I'd like to hear what other people experienced.

I'm curious how the VPN handles this.  I use my VPN very minimally now, esp since I work from home.  Might have to do some testing if I get a chance. 

[mimugmail]

We will track this a bit .. but please dont flood this issue:
https://github.com/opnsense/core/issues/5387#issuecomment-984432785