Since I am not sure if its a bug or feature this post, maybe others have seen it before:
I have some opnsense firewalls connected to the same /24 WAN subnet.
Firewall A: 212.x.x.1
Firewall B: 212.x.x.2
Router: 212.x.x.254
+------------------+
| Router |
| 212.x.x.254 |
+--------+---------+
|
|
|
|
| 212.x.x.0./24 WAN
+--------+-------------------------------+----------+
| |
| |
+--------+---------+ +----------+---------+
| Firewall-A | | Firewall B |
| 212.x.x.1 | | 212.x.x.2 |
+--------+---------+ +----------+---------+
| |
| LAN A | LAN B
| |
| |
+-----+-----+ +-----+------+
| PC | | PC |
| 01 | | 02 |
+-----------+ +------------+
The availability from the LAN of Firewall A to the WAN Interface of Firewall B looks like this:
(https://forum.opnsense.org/index.php?action=dlattach;topic=21425.0;attach=15103;image)
After doing a traffic capture on Firewall A and B I think I found the problem. Firewall B does not send the Traffic directly back to Firewall A.
The ARP traffic is sent to a combination of IP of Firewall A but with the MAC of Router. I reviewed the ARP table on Firewall B but there the entry was shown correctly.
We replaced hardware and reinstalled but problem persists with multiple installations and different firewalls on the same WAN interface.
For testing I changed:
net.inet.ip.redirect 1
net.inet.icmp.drop_redirect 0
-> no change.
Firewall->Settings->Advanced->Disable force gateway
-> no change
QuoteFirewall->Settings->Advanced->Disable force gateway
try to disable reply-to also
(dont forget to kill states after)
Thank you, that seems to fix it. I found an older long discussion https://forum.opnsense.org/index.php?topic=15900.0