NAT Missing when failing back from a backup wan to the main wan (Solved)

[hgerding]

this is for information only, as I "seem" to have fixed it. But it was enough work that I would like to pass back what I did to fix it.

I have a dual internet connection with one the "main" connection having a static IP and the secondary having DHCP

MAIN WAN
Static

Secondary WAN
DHCP

When I failed over the connection (by unplugging the wan) it failed to the secondary wan properly as well as 2 VPN connections to other sites. (Worked Well)

When failing back it would not switch to the "MAIN" connection properly, my first mistake was not setting up a proper failover group

Added a SYSTEM-GATEWAYS-GROUP
In the group I set
MAIN WAN - Tier 1
SECONDARY WAN - Tier 2
Trigger - Packet loss
(I also did some tunning of the packet loss in the gateway settings)

It still would not fail back properly
Now some where along the way my SYSTEM-GATEWAY-LOGS was generating an error message about a duplicate MAIN/Wan gateway. My way of fixing that is to erase the gateway and then check the settings on the one that appears after you delete it. (Which explains my disappearing settings on my gateway)

Then I retested the failover and again I got it to fail but when failing it back to the main connection it did not work.

This time I found that the FIREWALL-NAT-OUTBOUND was missing all of the NAT rules for the "MAIN" wan connection. I noticed that the Secondary WAN had the MAIN interface as part of its NAT rules. So I looked at the interface and found that the INTERFACES-MAINWAN had the  IPv4 Upstream Gateway set to auto. I switched it to the correct gateway.

Then I went back to FIREWALL-NAT-OUTBOUND and it had now generated the outbound nat rules for the main interface. (Nothing I had done to this point would auto-create the NAT rules.)

I hope this helps someone else.