IPsec tunnels dies after a few hours, but tunnels status is still up on OPNsense

[Gilad]

Started after installing the latest version - 23.1.4 (and still happened after 23.1.4_1).

The two IPsec tunnels were rock stable with the previous versions.

The tunnels status is still up on the OPNsense GUI, but I can't ping or SSH to hosts on the other side.
After around 40 minutes I'm getting these entries in the log:

2023-03-27T13:13:02   Informational   charon   05[ENC] <con3|25> generating CREATE_CHILD_SA request 8 [ N(REKEY_SA) N(ESP_TFC_PAD_N) SA No KE TSi TSr ]   
2023-03-27T13:13:02   Informational   charon   05[IKE] <con3|25> establishing CHILD_SA con3{57} reqid 3   
2023-03-27T13:13:02   Informational   charon   05[KNL] creating rekey job for CHILD_SA ESP/0xec8f744e/167.xxx.xxx.xxx

and then this:

2023-03-27T13:16:11   Informational   charon   11[IKE] <con2|24> rekeying IKE_SA failed, peer not responding   
2023-03-27T13:16:11   Informational   charon   11[IKE] <con2|24> giving up after 5 retransmits   
2023-03-27T13:15:47   Informational   charon   11[KNL] <con3|25> unable to delete SAD entry with SPI c088053f: No such process (3)   
2023-03-27T13:15:47   Informational   charon   11[IKE] <con3|25> giving up after 5 retransmits

Only restarting the strongswan service solves the problem.

Any ideas?