Multi-WAN (3): 2+1 Failover & Priority

[Kallex]

On WAN side I have a of two wired connections (Cable and Fiber) and one 4G/LTE failover.

Let's call them:

WAN_CBL (1Gbps down, 100 mbps up)
WAN_FBR (500 Mbps down, 500 Mbps up)
WAN_FO4G (~40-50 Mbps down, ~10 Mbps up)

On LAN side I have 4 logical groups of machines/devices (separated by routing needs):

LAN_Desktops
LAN_Mobile_&_IoT
LAN_Servers
LAN_RemoteWorkClients

VLANs are used to separate (some) parts, but preferably IP/Groups can be used to "identify device group". LAN is having fully managed VLAN capable switches.

I want to dedicate certain local groups to specific WAN, but failover to other wired and eventually to 4G. So failover priority being like:

WAN_FBR => WAN_CBL => WAN_FO4G
WAN_CBL => WAN_FBR => WAN_FO4G

When everything is up and running the allocation of WANs would be as following:

WAN_FBR:
LAN_Desktops
LAN_Servers (* having priority/guaranteed minimum bandwidth over Desktops)

WAN_CBL:
LAN_Mobile_&_IoT
LAN_RemoteWorkClients (* having priority/guaranteed bandwidth over Mobile & IoT)

So to the questions:

1. Is it possible to have failover-fallback between WAN_FBR & WAN_CBL before eventually ending up to FO4G?
- So that the WAN_FBR and WAN_CBL both are under "only functional wired wan" if one is still functional

2. If the failover is possible, is it possible to allow "fallback of priority groups" also on failover
- LAN_Servers and LAN_RemoteWorkClients preferably keep their bandwidth shares/relative shares or priority

3. To keep Servers available - DDNS is to be used (on failover), but can be done outside the OPNsense
- This would be nice bonus, DDNS is on Route53, can be solved using device's own IP detection (as is now)

The OPNsense is running on DEC840 currently with all WANs having port of their own (3) and LAN having one port. In the future possibly moving to use SFP+ ports for 1Gbps+.

I think/believe/hope the above is doable, but being new to OPNsense, I don't have clear understanding what to group and how to define failovers properly.

[muchacha_grande]

Hi Kallex,
for your first question, you need to define 2 gateway groups with three tiers each, indicating the gateways in order on each tier.
Then with policy routing you can select which gateway group is used based on IP address or LAN segment or whatever criteria you choose.
You can find the docs here: https://docs.opnsense.org/manual/how-tos/multiwan.html

For your second question, have a look at traffic shaping: https://docs.opnsense.org/manual/shaping.html
I really don't know if you can combine traffic shaping with gateway groups, but you can try.

For your thrird question you will need to define you DDNS in services >> dynamic dns, but again, you need to try if you can pass your active gateway IP to your domain name.
In this case you could use DNS-SRV key types on the DNS server. This way you could define your services with priorities on the DNS side and it should answer queries based on the priority and availability, so when you loose a WAN the DNS server would know it and pass an alternate IP address on queries.

[Kallex]

Thank you for prompt and detailed reply! This pushed me indeed to proper direction and helped a lot with the initial confusion of various bits and pieces.

Traffic shaping is likely not a problem (I mean if I can't get it to work), as we have plenty of bandwidth, but I will play around if I can get traffic shaping to run on top of gateway groups.

I'll play around and update the post with results once done.

[muchacha_grande]

If you use shared forwarding, that I believe is enabled by default, you will have traffic shaper and policy routing.

https://docs.opnsense.org/manual/firewall_settings.html#shared-forwarding

I've not used both simultaneosly but if you play around a bit you can make it work as intended.

[Kallex]

Thank you for that additional detail.

I managed to do the important parts 1 and 2 of the above. My fiber isn't yet setup, so the final setup is waiting to be tested (now "faked" the fiber partially in the configurations).

The traffic shaping with those bandwidths and small amount of comps in total (max 5 transferring simultaneously) would still leave ~100mbps for each just "fairly distributed", so I went with easy setup with pipes & queues to achieve just that.

I'll come back still to update the post, once the fiber finalizes and I can properly conclude the real testing. Should be somewhere next week hopefully !

[Kallex]

Got the Fiber now and the 2+1 WAN was set up basically as described here. Failovers work nice, DDNS is only part remaining and its trivial with the options available.

Haven't set/tested the server/port forwarding behavior on failovers, but with the well working setup up to this level, I don't expect any surprises there.

Thanks a lot for pointing me to right direction.