"
I can confirm this issue. Is there any progress or better workarounds?
ISSUE
--------------------------
In my situation, I define multiple non-upstream gateways with different priorities in Gateways -> Configuration:
GW100 192.168.1.100 upstream=no priority=100 monitor=10.0.0.100
GW101 192.168.1.101 upstream=no priority=101 monitor=10.0.0.101
...and then I add the same route for each gateway in Routes -> Configuration:
10.0.0.0/8 via 192.168.2.100
10.0.0.0/8 via 192.168.2.101
The expectation is that the system routing table will populate with a route for whichever gateway is up with the highest priority (in this case, it would be GW100). Instead, opensense seems to randomly select which gateway will get the route in the system table, regardless even of whether the gateway is up or down.
This issue seems to only apply to non-default gateways. Default gateways seem to handle this correctly.
WORKAROUNDS
--------------------------
1. Like zubrick said, you can manually create more precise routes for the preferred gateway to force opnsense to put both routes in the system table. However, if the preferred gateway goes down, opnsense doesn't update the routing table is not updated to remove the applicable route(s). Rather, you have to manually disable the preferred gateway to remove the route, which is not optimal.
2. Can't use gateway groups in NAT rules because the traffic is not being NATted.
3. Can't use gateway groups in Routes -> Configuration, because opensense does not allow groups to be the target of route rules.
4. Can create a gateway group and then create firewall rules for all interfaces that force the gateway for traffic going to 10.0.0.0/8, but this would mean duplicating every firewall accept rule (one for local traffic, one for remote traffic). I'm not sure if maybe there is a smarter way to do this with non-quick rules or marking?
5. Maybe some kind of monit scripting that can watch the gateway status and add/remove routes as needed? I couldn't find much documentation on it.
None of these options are great.
PREFERRED RESOLUTION
--------------------------
The two easiest ways to solve this from a user's perspective would be
(1) allow gateway groups as targets for static route configuration or
(2) have an option to treat down gateways as disabled for purposes of system routing table generation.
ISSUE
--------------------------
In my situation, I define multiple non-upstream gateways with different priorities in Gateways -> Configuration:
GW100 192.168.1.100 upstream=no priority=100 monitor=10.0.0.100
GW101 192.168.1.101 upstream=no priority=101 monitor=10.0.0.101
...and then I add the same route for each gateway in Routes -> Configuration:
10.0.0.0/8 via 192.168.2.100
10.0.0.0/8 via 192.168.2.101
The expectation is that the system routing table will populate with a route for whichever gateway is up with the highest priority (in this case, it would be GW100). Instead, opensense seems to randomly select which gateway will get the route in the system table, regardless even of whether the gateway is up or down.
This issue seems to only apply to non-default gateways. Default gateways seem to handle this correctly.
WORKAROUNDS
--------------------------
1. Like zubrick said, you can manually create more precise routes for the preferred gateway to force opnsense to put both routes in the system table. However, if the preferred gateway goes down, opnsense doesn't update the routing table is not updated to remove the applicable route(s). Rather, you have to manually disable the preferred gateway to remove the route, which is not optimal.
2. Can't use gateway groups in NAT rules because the traffic is not being NATted.
3. Can't use gateway groups in Routes -> Configuration, because opensense does not allow groups to be the target of route rules.
4. Can create a gateway group and then create firewall rules for all interfaces that force the gateway for traffic going to 10.0.0.0/8, but this would mean duplicating every firewall accept rule (one for local traffic, one for remote traffic). I'm not sure if maybe there is a smarter way to do this with non-quick rules or marking?
5. Maybe some kind of monit scripting that can watch the gateway status and add/remove routes as needed? I couldn't find much documentation on it.
None of these options are great.
PREFERRED RESOLUTION
--------------------------
The two easiest ways to solve this from a user's perspective would be
(1) allow gateway groups as targets for static route configuration or
(2) have an option to treat down gateways as disabled for purposes of system routing table generation.
