Gateway groups and NAT : Incorrect NAT IP used on interface.

[namezero111111]

Dear folks, we are debugging a strange occurrence in 18.1.6 with load balancing and outbound NAT.

When the machine is newly booted,  the wrong NAT IP is used.

Example:
DMZ1 192.168.4.131 with CARP 192.168.4.135
DMZ2 192.168.4.141 with CARP 192.168.4.150

DMZ1 and DMZ2 gateways are in a gateway group.
NAT 192.168.0.0/16 on DMZ1 via 192.168.4.135
NAT 192.168.0.0/16 on DMZ2 via 192.168.4.150

Pings time out/website won't load, etc...
tcpdump capture on DMZ2 show packages originating from 192.168.4.135 (DMZ1 CARP IP)

When NAT rules are reversed in priority,
tcpdump capture on DMZ1 show packages originating from 192.168.4.150 (DMZ2 CARP IP)

When "Sticky outbound NAT" is disabled, the problem disappears. However, is this setting not required for i.e. banking websites?
is there another, better way for load balancing by source except multiple firewall rules?

The problem persist when translating onto Interface IP rather than CARP IP.

Has anyone experienced anything like that?
I couldn't find anything related to that anywhere on here or github unfortunately.


Any suggestions?

[namezero111111]

After further research:

The issue occurs regardless of:
1. Source tracking timeout
2. Firewall tracking normal or conservative
3. Whether the NAT IP is CARP or interface address

The issue does NOT occur if:
1. Shared forwarding is disabled.

Opened https://github.com/opnsense/core/issues/2376 due to suspicion in shared forwarding.