OPNsense 19.7.7-amd64 // Multi-Wan Fail-Over: UDP working, TCP not

[hjint]

Hallo everyone

I have setup Multi-wan according to the documents in Wiki.

WAN1 & WAN2, both priority 255, are identical except for IP & DNS addresses
WANGROUP have WAN1 as TIER1 and WAN2 as TIER2 with Packet Loss & High Latency
All Lan rules point to WANGROUP as gateway except for the proxy & DNS rules
Sticky Connections is ON, Shared forwarding & Disable force gateway is both OFF
WAN1 & WAN2 NIC's are TP-Link Gigabit PCIE TG-3468 & both routers are Huawei B618

When WAN1 goes down, UDP services like VOIP & VNC switch to WAN2 and continue to functioning, but TCP services like mail and web browsing stop to function. What I have found by accident is that when WAN1 is down and the WAN1 network cable is unplugged, all TCP services start to function via WAN2 and when the WAN1 network cable is plugged back, TCP services stop to function.

Is there a setting that I miss somewhere or a rule that I need to create?

Any pointers will be highly appreciated, thank you in advance

[mimugmail]

Running connections wont switch if it's not really down. If you really need this you can clear state table after switch in Firewall : Settings : Advanced

[hjint]

I'm referring to LTE connections which is down completely, Loss=100% & No LTE signal

Disable State Killing on Gateway Failure is OFF

System: Routes: Status show default destination gateway as WAN2 IP address when the fail-over occur

[mimugmail]

Does this also happen after around five minutes? Maybe something timeout related?

[hjint]

WAN1 is down for just over 13 hours now and when I connect the WAN1 cable, no web or email, when I disconnect the WAN1 cable, web & email is available. I have done a cold restart of the box about 8 hours ago with no effect.

Have connected the WAN1 cable and left for 35min without any effect, after disconnecting the cable, web and email are connecting again

[mimugmail]

Screenshot of gateway monitoring please

[hjint]

These screenshots are with WAN down and WAN cable connected

[hjint]

These are with the WAN cable disconnected

[hjint]

This issue resolved itself either with the upgrade to version 20 or with the re-run of the General Setup Wizard or the combination of both.