Issues with routed IPSec (VTI) on HA-Firewalls

[Ketanest]

Hi together,

I hope this forum is the correct one even if there is HA involved.
We've got the following problem: We set up some VMs to test OPNsense in our business environment. We set up IPSec tunnels between 3 of our locations. Routing is done by OSPF and routed ipsec (VTI). That works fine. No we got two more locations where we decided to implement OPNsense with HA. That are two physical servers per site, configured in HA-mode. We have an /29 subnet for WAN at each site so each node has an own public IP and we configured CARP to share a VIP. The configuration between HA clusters and the VMs is completely the same but we have a problem with the site-to-site tunnels to the HA clusters. The tunnels regularly break and do not come up again or come up hours later. When the tunnels are down the log is full of entries:

Code Select Expand

querying policy 0.0.0.0/0 === 0.0.0.0/0 in failed, not found
Then we have to restart the tunnel on one of the sites (HA or VM). Service restart is usually not necessary.

Config of the tunnels (is on each node the same):

Code Select Expand


P1:
Connection method: start on traffic
Key Exchange: IKEv2
Internet Protocol: IPv4
Interface: WAN Interface or VIP
Remote Gateway: Public IP of WAN Interface or VIP
Auth method: Mutual PSK
Identifiers: Public IPs
PSK: PSK
Encryption algorithm: AES256
Hash: SHA512
DH key group: 14
Lifetime 28800

Install policy: no
Disable rekey: no
Disable reauth: no
Tunnel Isolation: no
SHA256 96 bit trunc: no
NAT Traversal: Enable
Disable MOBIKE: no
Close Action: none
DPD: on, 10 seconds, 5 retries, action: restart
Inactivity timeout: -
Keyingtries: -
Margintime: -
Rekeyfuzz: -

P2:
Mode: route based
Local address: unique IP in an /30 net
Remote Address: other unique IP in the same /30 net
Protocol: ESP
Encryption algorithm: AES256
Hash: SHA512
PFS key group: 14
Lifetime: 3600
Automatically ping host: -

EDIT: What I also recognized is that sometimes an entry with connection name "((unnamed))" appears (attachment). The local IP and remote IP are the same as in the real connection.

Has anyone an idea what the problem could be here?

Thank you!
Ketanest