Traffic leaves WAN on wrong interface

[Ketanest]

Hi altogether,

I have the follwing problem:
I have configured 2 WAN interfaces and gateways. One is an LTE connection (with an LTE-Router with fixed IP and /29 net) the other a cable connection with dynamic IP and extra router.
An IPSec tunnel is configured on the CARP address of the LTE interface.

Default gateway is on the cable connection. The problem is: any WAN traffic leaves the firewall on WAN_Cable interface via the default gateway 192.168.178.1 even though the IPSec tunnel is configured on WAN_LTE carp (it has neither worked with setting tunnel direct to WAN_LTE interface). Incoming connections e.g. on the webinterface via WAN_LTE interface are also answered via WAN_Cable interface. Firewall settings are default, especially "Disable force gateway" is disabled so usually the firewall should answer or send packets from the WAN_LTE interface via the WAN_LTE gateway but it does not.

Config:

Code: [Select]

OPNSense Version: 22.4.3_1

Interfaces:
WAN_LTE: 82.x.x.4/29
WAN_Cable: 192.168.178.10/24
CARP WAN_LTE: 82.x.x.3/29
CARP WAN_Cable: 192.168.178.12/24

Gateways:
WAN_LTE_GW: 82.x.x.1/29 on WAN_LTE interface prio 255
WAN_Cable_GW: 192.168.178.1/24 on WAN_Cable interface prio 250

Outbound NAT rules:
WAN_LTE and WAN_Cable interfaces (each rule for each interface):
This firewall / any source -> any / tcp/udp 80,443,123,53 -> translation to interface address
This firewall / any source -> any icmp -> translation to interface address
This firewall / source tcp 4444 -> any -> translation to interface address (webinterface is on port 4444)
any -> any -> translation to interface belonging CARP IP

The first NAT rules are to be able to do updates, DNS,  NTP and reach webinterface on each node in a HA setup.

With a static route to the IPSec endpoint via the LTE-Gateway the tunnel works fine but that should not be the solution.

Can you help me here?
Thank you!
Regards
Ketanest

[Ketanest]

Found the problem but have no solution:

We have a setup with opnSense business edition and do our firewall rules only on floating tab because our firewalls have different interface names and order so we can not make sure that the rules would be applied to the correct interface. Reply-To is the thing we'd need but that works only on rules on WAN interfaces (floating does not match this requirement). Since we manage our rules centralized we would overwrite the WAN interface rules if we'd configure some rules manually. So we have to proceed with this workaround (static routes to the endpoints).

EDIT: Not completely the problem. This is the problem for packets arriving on the firewall. Packets originating from the firewall should be correctly routed...

Regards
Ketanest