NAT issue

[bigops]

I had posted this in the 22 forum earlier.  https://forum.opnsense.org/index.php?topic=31961.msg154477#msg154477

The issue with outbound NAT seems to still persist in the 23 version also.  The issue is that if there is a gateway group with dual WAN interfaces in it and for operational reason a specific outbound traffic is redirected to a gateway with a lower priority (other than the gateway group) sometimes the outbound traffic seems to land up on the wrong gateway.  Rebooting the appliance does not seem to solve the issue, but manually clearing the state table again puts the traffic onto the correct gateway. 

This used to work fine in all earlier versions so seems to be some kind of bug introduced recently.

Skip rules when gateway is down is checked to prevent gateway rewrite on failure.

[sorano]

Yeah I'm seeing the same behaviour.

Had a failover occur and traffic did not not switch back to primary gateway even though it was up.

[bigops]

Has this been observed by anyone?  The issue is becoming more frequent and I have to reset the table every couple of days for this to keep working.  Is this a bug introduced in OpnSense / FreeBSD?

[voideris]

I am really not sure if this issue is affecting me but symptoms do look like it can.

I have WAN1 and WAN2, with multiwan failover and vpn network that should be routed only through WAN1.
What I observed was really flaky VPN behaviour after update to 23.1. I tried to diagnose it but I have not really change anything from last 22.7 version and it was rock solid before the update (Wireguard).

I assume the problem flow would be something like that:

• Failover  from WAN1 to WAN2.
• Some time when routing is still working.
• WAN2 fails.
• Routing is stuck with WAN2 (conjecture).
• I notice problems with VPN connections
• Gateway monitoring shows the same status for VPN gateway and WAN2 (packetloss/red/dead).
• Restaring VPN does not help, sometimes changing wg peer helps but flakes out really quickly.
• I need to reboot opnsense box.

From what I remember, when I configured VPN it should really only use WAN1 connection and in case of failover to WAN2 just die. I am not sure it ever worked that way but before current version (23.1) I have never had such issues.

This behaviour is persisting since my update to 23.1, so since few days after it went up for download. I did not do clean install, I updated through web interface.

So I am stuck rebooting the router every few days.