HA sync not working

[Mathwest]

I have searched arround a lot and have a quite hefty setup as well as multiple setups of PFsense with HA (We are moving away from PFsense) but for some reason all i am met with is the in my opinion rather bad and non descript message of "The backup firewall is not accessible or not configured"

Here is what i have done and our setup.

Underlying is xcp-ng, all external networks come in as a trunk and vlans handled in opnsense.
sync network makes use of xcp-ng XOA SDN and is an tunnel network.
I have configured the HA settings as documentation states.
Master has sync enabled, target ip is the ip of the backup. i have dubble and tripple checked IP, user and password is correct.
i have an allowed all rule on the syn interface.
backup has sync enabled and only an peer ip, no other configurations.
Firewall has allow all on sync interface for backup aswell.
i have even tho CARP is not set up yet even made sure to allow carp on all interfaces as a floating quick rule.

I can ping between the two devices no issues.
I can see pfsync traffic in the packet capture on the interface of the backup and i can see it replying. (small snipet of this traffic in the attachments)

No matter what i do i cannot get it to go past this non descriptive error message because the backup is most definitively configured and reachable even if it states it is not.

I am on 24.1.1