Dual WAN issues

[Heathy65]

I have OPNSense running on a Pico PC (Intel® E3845 4xLAN) and it's been rock-solid for months with one very big notable exception.  This is relating to my dual WAN setup.  I have 2 x VDSL circuits (both providing c.20Mbps downstream) both of these services are using the providers original routers and these two units are connected upstream to 2 x Ethernet interfaces on my OPNsense.

When I have a single gateway in OPNsense enabled (either WAN1 or WAN2 doesn't matter), everything is rock solid.  However if I configure any load-balancing then things seem to get unpredictable and unreliable, i.e. loss of Internet connectivity.  I've even had some issues when I have OPNSense configured for just failover.

I even resorted to using each of the single broadband connections each from their local subnet as a test (so no OPNsense) and both circuits are stable, no errors and long up times.

In OPNsense, I've double checked my configuration but I'm thinking I must have done something stupid...

My gut feel is it's something to do with the "Monitor IP" configuration on the gateways.  In that regard I have tried using different IPs for the monitoring (I originally used 8.8.8.8 & 8.8.4.4).  I even tried disabling this yesterday and that didn't seem to help.

I have even changed my downstream WAN2 router to be in bridge mode (thus eliminating the WAN2 router) so now I have OPNsense configured for PPPoE.  In this configuration the gatway is showing Down in OPNSense but it is actually up and working.

Any thoughts/guidance appreciated.

[Praelia7or]

I had a similar issue on my old Asus router, lots of services don't like having connections from different IP's, website acted weird/slowly, services were slow or failed to connect - everything was fine on failover though.

There is a workaround for this, I remember something like a persistent connection setting - it may be worth searching for this in settings/google. I didn't look into it further as load balancing isn't ideal for my setup.m

[va176thunderbolt]

Under Firewall, Advanced, Multi-wan you'll need to enable "Sticky Connections". You'll also probably want to set a reasonable value in the timeout - say 10 or 15 minutes.