Skip rules when gateway is down or gateway monitoring bug?

[user1234]

I need to set the gateway that each interface can use and this needs to be failsafe so I have enabled "Skip rules when gateway is down".
However I find that none of my traffic is reaching the gateway, it appears that the firewall thinks the gateway is down as no rules are created and all my traffic hits the default block rule. If I disable "Skip rules when gateway is down" then all the traffic goes out the default firewall gateway instead of the one I have specified. When I check the gateway under system->gateways all the gateways are marked as online (although gateway monitoring is disabled).

My gateway is not a physical interface but a openvpn client with a virtual IP.
Any idea why this is happening or how to debug this problem?

Edit: I am running 18.7.9