Thank you for the prompt reply! I was struggled to search for anything meaningful. I think it is asymmetrical routing. While I am not sure what that is exactly, changing the state type to none fixed the issue. I found the information on this forum:
https://forum.opnsense.org/index.php?topic=9136.msg40997#msg40997
https://forum.opnsense.org/index.php?topic=9136.msg40997#msg40997