Connection established via allowed rule, then magically denied via default deny

[Bubba88]

OPNsense 22.7.10_2-amd64
FreeBSD 13.1-RELEASE-p5
OpenSSL 1.1.1s 1 Nov 2022

I noticed that establishing an SSH session from one host to another works, then in a small period of time (less than a minute) the connection is stopped, from the perspective of the user using ssh.  In the OPNsense logs I can see the behavior but I don't understand why.  I know this was working a month ago (or so) and now this behavior.  I've rebooted the "firewall machine" (it is a KVM running on a proxmox hypervisor system). 

As you can see in the attached image, when the connection is established the rule is triggered (green entry), then after a period of time the defauly deny rule starts being applied.

I'm hoping someone can point me in the right direction.  This is very annoying and nearly impossible to work around. Any ideas what is wrong?

[Patrick M. Hausen]

Asymmetrical routing violating the firewall state. The packets from the server to the client do not travel the same way back as the packets from the client to the server.

Network diagram and all IP addresses involved, please.

[Bubba88]

Thank you. Hopefully I'll find the routing issue as I investigate and attempt to document what I've created to post here. Except for my physical LAN, everything else is virtual using Proxmox (containers and KVM) and Openvswitch.  I've only noticed the problem between LAN and the VM environment. Some containers are dual homed, but only the OPNsense KVM should be routing packets.