IPsec and State Synchronization - unexpected behavior

[Monviech (Cedrik)]

I had this weird behavior between two OPNsense in HA while using IPsec (between DEC hardware and VM with pcie passthrough, all interface names are the same and theres a lagg)

Quite often, I connected an ikev2 ipsec tunnel, phase 1 and phase 2 were up, but there was no traffic from the opnsense back to the remote peer. It always worked the first time, but the second time it didn't. This behavior mostly affected roadwarrior connections with lots of reconnecting and lesser the site2site tunnels.

My troubleshooting led me to State Synchronization. In Sessions I could also see established TCP sessions even though the tunnel was down.

When I deleted the IP Addresses of the traffic selector (e.g. 192.168.0.0/24) from the state table on both firewalls and restarted the ipsec tunnel, it worked again with Tx and Rx.

To mitigate this behavior:
I created extra firewall rules in Firewall: Rules: IPsec which timed out TCP faster (after 600 seconds).
Then I disabled state syncronisation by setting the "State Type / NO pfsync" parameter for all rules in Firewall: Rules: IPsec.

I didn't come to a conclusion, I just know that my mitigations work and all roadwarriors can connect every time now. It would be interesting to know if that's an expected problem between hardware and vm, or if it could theoretically happen between two hardwares too.