Dual wan setup not working

[gica78r]

Hi everyone. This is my first post on the forum but I can't find an answer in older posts.

I'm experiencing a strange issue. This is my setup:

Software: OPNsense 18.7.10_3-amd64
Hardware: PC Engines APU 1d4
Wan configuration

• wan1: pppoe0 (re2_vlan666)
• wan2: pppoe1 (re2_vlan667)

So my two gateways are configured over two logical vlan interfaces, both with pppoe. I have a gateway group called WAN, where wan1 is the main gateway (tier 1), while wan2 is the backup gateway (tier 2). The DNS servers are configured in the right way, one server per gateway (I'm using OpenDNS servers: 208.67.220.220 and 208.67.222.222). To monitor the status of the gateway I'm using the Google's DNS server addresses (8.8.8.8 and 8.8.4.4).

Issue description: if both gateways are online, all works fine and wan1 is the default route:

Code Select Expand

root@apu:~ # netstat -r
Routing tables

Destination        Gateway            Flags     Netif Expire
default            192.168.100.1      UGS      pppoe0
8.8.4.4            10.4.55.92         UGHS     pppoe1
8.8.8.8            192.168.100.1      UGHS     pppoe0

....
....
208.67.220.220     10.4.55.92         UGHS     pppoe1
208.67.222.222     192.168.100.1      UGHS     pppoe0


If the main gateway goes offline, the backup link works for a few seconds (I can see my public ip change) and then the firewall stops forwarding traffic; the second gateway stays online, but the firewall misses a default route:

Code Select Expand

root@apu:~ # netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
8.8.4.4            10.4.55.92         UGHS     pppoe1
....
....
208.67.220.220     10.4.55.92         UGHS     pppoe1


This issue happens both with active/standby wan links and active/active (load balancing) wan links. I tried to configure one of the gateways as the default one, bot nothing changed.

It can be possible that this issue is due to the fact that I'm using two logical interfaces on the same physical interface for the wan links? Has anyone had the same problem?

Thank you in advance!