Multiwan conection issue / dpinger latency

[cbesener]

Hello,

I am having an OpnSense 20.1.4 with multiwan setup. One connection fibre channel and one connection as backup with LTE.

Sometimes I can read in the Gateway Log

2020-04-27T15:23:41   dpinger: GATEWAY ALARM: WAN_TMOBILE_DHCP (Addr: 8.8.4.4 Alarm: 1 RTT: 1030991ms RTTd: 2527732ms Loss: 0%)
2020-04-27T15:23:41   dpinger: WAN_TMOBILE_DHCP 8.8.4.4: Alarm latency 1030991us stddev 2527732us loss 0%

At this moment all traffic is interrupted, also VPN connections stock a view seconds. But the Alarm is only on the backupline. At this moment there is no problem on the fibrechannel.

Can anybody help me, why the connection is interrupted, if there is a problem on the second line?

[shadesh]

Same problem here with LTE as backup... the whole gateway has a short hicup when the Backup Line is dead or has reached the threshold.

https://forum.opnsense.org/index.php?topic=16666.msg76127#msg76127

[marjohn56]

It's because the firewall is flushed as all the existing connections going out to one gateway are removed. When it fails-over to the backup connection gateway new connection states are made. I see this quite often when I'm using something like NewsBin Pro, if I deliberately drop the one gateway then I get loads of warnings about the lost connection, after a few seconds it reconnects using the other gateway, smart software will try and re-connect.


Have you set the priorities on the gateways?

[shadesh]

I already tried with different priorities (255 on LTE backup and 254 on main line). No change in behavior when the LTE backup reached the threshold e.g. is offline.

Could be "Disable State Killing on Gateway Failure" a solution for that?

My problem is, i have a lot MQTT devices in different VLANs, every time the LTE Backup have a problem, the MQTT devices also disconnects because the OPNSense is the gateway and firewall for my VLANs at home.

Looks like there is already a discussion about this: https://github.com/pfsense/pfsense/pull/4159

[marjohn56]

Might well be a discussion, but that's pfSense, not Opnsense.

[shadesh]

Looks like for now "Disable State Killing on Gateway Failure" is a solution for that. I have to test what happens when my main line is down - Would be really nice to have such a feature only for the non active gateway.