Testing Multi Wan

[sirio81]

Hi all, I successfully configured multi wan.
Now I'm playing with it to check if it works as I expected, and guess what...it doesn't 

My gateway group is named 'wangrp' and has 2 gateways
isp1 Tier 1 (this is te default gw)
isp2 Tier 2

On system / gateways / all I set 'Mark Gateway as Down' on the the gw isp1.
I check my ip by 'dig +short @resolver1.opendns.com myip.opendns.com' and also by sites like ping.eu and I still see the isp1 public ip.

I was expecting to see isp2 public ip.

I remove the flag 'Mark Gateway as Down' and aplly changes and check the publick ip.
Now both gateways are up so I expect to exit with isp1 but checking the publick ip, I see isp2 ip.
The exact opposite.
on Gateway / Status they are both online.

I repeated the procedure disabling the the isp1 gw instead of marking it down.
This time the connection is correctly router through isp2 gw.
I re-enable isp1 gw but I still get routed through isp2 gw (Tier 2) instead of isp gw (Tier 1).
Note: I disabled Sticky Connection' on firewall advanced settings.

Am I missing something?

[mimugmail]

At first you should test from internal hosts against speedtest providers if you have the correct IP.
Also it would be good to test failover in a real way like plug the cable or block a connection to the monitoring IP.

If this works we can work on dyndns ...

[sirio81]

I checked by myip.com, ping.eu and other sites that shows my public ip.
I also checked form by linux pc querying opendns directly by command dig (no possible browser cache related problem there).

I also tested the real world scenario unplugging cables.

When I unplug isp1/Tier1 cable, both gateway go down, connections doesn't work off course.
When I unplug isp2/Tier2 cable, on the right gw goes down and connection works.

Note: "Trigger Level" is set to "packet loss or high latency".
I noticed it may behave differently if I set "Trigger Level" to "member down".
I'll report back after few more tests.

[sirio81]

Sorry guys, but has anyone been testing multi wan on opnsense 18.1?
Just to understand if I'm the only one having troubles!

I made a factory restore and started form scratch.
My configuration, as describe before, is a gateway group with 2 gw:
  isp1 Tier 1 (also default gw)
  isp2 Tier 2.
  Trigger Level = Member Down

1) I mark down the gateway Tier 1:
    the isp1 gw status dosn't change (stay online).
   I get the same behaviour if I unplug the cable.

2) I mark down the gateway Tier 1:
    the isp2 gw status dosn't change (stay online).
   I get the same behaviour if I unplug the cable.

3) If I disable the isp1 gw, the connection gets corretly switched.

Please, may anyone test this "simple" scenario with 2 internet connection in fail over configuration.

Note: I also took care to set monitor ip different from the dns i set in system / settings / general.
Note 2: it behave the same also with "Trigger Level" Packet Loss.

I think the problem i repeatable.
I would like to open a bug report but it would be great if someone else can confirm it first.

[mimugmail]

You dont have to file a bug and I think I've tested multi wan more then any other ..

https://github.com/opnsense/core/issues/1821#issuecomment-370360822

[sirio81]

Damn, I woke up this morning and I remembered i probably forgot to remove the flag "Disable Gateway Monitoring" after the re-configuration.
Today I'm not at work and I can't test it.
I'll do it the next week.
Thank you very much mimugmail for the support.

[elektroinside]

I'll soon jump in as well the multi-wan testing (on a regular basis).
I stupidly insist doing this with a second Miktrotik + a Huawei E3131 modem, for which I need an external antenna to stabilize the signal a little
Hopefully, I'll get one today.