Firewall failover STILL not working

[drivera]

Hi!  I've posted about this before (https://forum.opnsense.org/index.php?topic=11497.msg52045#msg52045).  The issue is still there: on a prolonged outage for the primary circuit (Cable), every so often the firewall's default gateway will simply get nulled out (i.e. set to "nothing") even though the secondary circuit (ADSL) is up and running.

The "workaround" is to log into the UI, open the ADSL gateway's configuration, save it (no changes!!), and then click on "Apply Changes". This causes the ADSL link to be selected as the default gateway.  But then again, a few minutes later, the same thing happens again (default gateway gets de-configured), and off we go again to the workaround...

Here are some configuration tidbits:

• There are 4 gateways in the system: Cable, ADSL (these are physical interfaces), VPN1 and VPN2 (these are "soft" interfaces - OpenVPN the both)
• I added all gateways to the same group, with Cable as tier 1, ADSL as tier 2, and the VPN gateways as tier 5
• The VPN interfaces are configured with "Mark Gateway as Down", precisely so they won't be promoted to primary (not that it matters if both Cable & ADSL are down)
• Both Cable and ADSL have explicit monitoring IPs set, in order to validate if the link is really up, vs just the interface is up (frequent case when Cable goes out is that the interface remains in the UP state, even though the actual link is down)
• All gateways are set for DHCP on IPv4
• NONE of the gateways is configured with "Disable Gateway Monitoring" as this will (erroneously, if you ask me) override "Mark Gateway as Down" and cause the gateway to be marked as UP even if you don't want it to

Basically, I have everything configured like the "textbooks" say I should have it, and yet I can't get it to work the way (I think) it should.  The problem seems to be with dpinger (or related processes), since if I change the VPN gateways to "Disable Gateway Monitoring" (i.e. assume they're always UP), then for some inexplicable reason they will be preferred ahead of the ADSL link as gateway, even though the ADSL link is in a higher tier within the same gateway group...!!!

Can someone please help me figure this out?

Thanks!

[Antaris]

I also have non-switching failover situation with 2 IPSs on fiber optic via media converters with public IP addresses.
The guide i used is:
https://wiki.opnsense.org/manual/how-tos/multiwan.html

[mimugmail]

Do you have default gw switching enabled on System : Settings : General?

[Antaris]

GW switching was not enabled. I now find this variable thanks to you. It's not mentioned in official guide.
Thanks a lot. Will check it on site.

[mimugmail]

You are very welcome to contribute this to the docs
https://github.com/opnsense/docs

[Antaris]

The system started to switching gateways with the help of @mimugmail. But when the primary ISP goes up again, the router not switching to Tier1 until Tier2 is not failed (in my case Tier1 is ~1gbps, Tier2 is ~150mbps).
About editing the docs via github i will try, but i am new it github too.

[drivera]

Do you have default gw switching enabled on System : Settings : General?

In my case, this setting has always been on, and I still have this issue. In fact, I just made another post about it providing a bit more info since this thread was sort of stale...

[mimugmail]

Link?

[Antaris]

Link?

https://forum.opnsense.org/index.php?topic=15554.0