IPsec + VoIP with same ISP: IPsec site-to-site with one common IP for a host

[blblblb]

So, I will describe the scenario a little bit:

• Site to site IP sec tunnel, both sites have the same VOIP vlan provided by the same ISP (they have the same provider for fiber uplinks)
• ISP enables a specific A.B.C.D IP for a SIP proxy in the VOIP vlan. This is the same for all clients. Even the subnet and DHCP leases inside the vlan are the same.
• SIte A should be able to access SIte B's A.B.C.D so phones and other PBX systems can use the trunk for site B over the IPsec tunnel.

Site B uses 10.1.x.x subnets for most internal networks, SIte A uses 10.2.x.x... (just to simplify things, not the actual config). Site B has no migrated away from pfsense so it still runs pfsense 2.5. Site A is Opnsense.

Things I have tried, while admittedly feeling stupid, to no avail:

• Virtual IP 10.44.44.44 at SIte B, tunnel for 10.44.44.44 and the remote subnet set to Site A's VLAN subnet (let's say 10.2.56.0).
• NAT 1:1 and port forward for TCP/UDP/ICMP (all ports) from 10.44.44.44 to the A.B.C.D SIP proxy IP in SIte B's VOIP vlan.

(and the FW rules at SIte B to allow for A.B.C.D to be explicitly routed from the VOIP vlan gateway)

Hitting a wall, so any help appreciated. If you can throw a recipe at me, that also works (but I do want to understand the solution ;P).