Firewall Rules: Change gateway depending on destination address

[marcoh00]

Hi there!

I started using OPNsense a few days ago as manually managing iptables rules got too complex for me and I wanted something nice and managable - looks good so far :-)

Unfortunately I'm having difficulties with my setup which is probably quite exotic! This setup is for my home server and unfortunately my ISP doesn't offer me a public ip address (CGNAT only), everything I get is an /64 ipv6. Having said that, I bought myself a VPN connection which offers a public ipv4 address and works over OpenVPN over IPv6. I have successfully managed to set it up and it seems to work, as I can ping other hosts through it and I receive packets. The third interface is a private network for my server virtual machines. All in all my OPNsense setup looks like this:

WAN (Private IPv4, Public IPv6) ----> ISP Router (192.168.178.0/24)
PublicIP (Public IPv4, Public IPv6) ---> Internet (OpenVPN)
VMnet (Private v4+v6, OPNsense does DHCP) ----> Servers (192.168.10.0/24)

As the OpenVPN provider does rate limiting I'm hesitating to set it up as my default gateway, I'd rather stay with my ISP for that. So what I want to do is to use WAN as my default gateway but whenever something is a reply (for example sent from my webserver) from VMnet, route it through PublicIP.

What I've done and tried:
- Set Outbound NAT manually on PublicIP interface
- Configure Port Forwarding from PublicIP/443/tcp to a host inside VMnet
- Added rule on VMnet interface which matches source port 443 from VMnet to any (for testing, after that I want two of these rules: one for 192.168.178.0/24 and another for every other destination address) and set PUBLICIP_GW_V4 as the gateway.

What works:
- Server vm receives packets and answers

What does not work:
Receiving those answers anywhere ;-) I have no idea how I could check what OPNsense does with the answer. Firewall/Live view was not very helpful unfortunately as it only shows the incoming packet forwarded to VMnet host. I think (maybe?) OPNsense tracks the state of the connection and never evaluates my rule on VMnet interface, so it tries to route the packet, which is answering the request from PublicIP, arrived on VMnet back via WAN, which will of course fail. Maybe I could tell OPNsense somehow to stop this so the rules on VMnet get evaluated and the correct gateway is chosen?

I would be very grateful for any tips regarding this. Thank you very much!