State tracking issue with OPNsense in single-interface public IP setup

[pefen]

I have the following setup:
- Proxmox host with a /29 public subnet (148.251.196.80/29)
- OPNsense VM: 148.251.196.85, gateway: 148.251.196.80
- Web servers VMs: .81, .82, .83 with gateway set to .85 (OPNsense)
- Single WAN interface, no LAN

The goal is to have all traffic from web servers pass through OPNsense for filtering before reaching the provider's gateway.

Issue:
- With state tracking set to "keep state", traffic gets blocked with "Default deny / state violation rule"
- Only works when state tracking is set to "none"
- Floating rules don't help
- Even with global setting "Firewall Optimization" set to "conservative"

Basic connectivity works (ping, web traffic) but I'd like to understand:
1. Why state tracking doesn't work in this setup?
2. Is using "none" state tracking a secure approach?
3. Is there a better way to achieve this?

Thank you for any insights.

[Patrick M. Hausen]

The gateway 148.251.196.80 will send request from clients to the web servers .81, .82, .83 directly. The webservers will send the answers to OPNsense.

OPNsense not having seen the initial SYN packet of the TCP connection setup and now seeing a SYN/ACK without a preceeding SYN, will drop the packets for a state violation.

You should place your web servers in a private network behind your OPNsense.

[pefen]

Thanks for the explanation. Since moving servers to a private network is not an option (due to Enhance control panel requirements for public IPs), is there a way to handle this asymmetric routing situation?
Specifically, can OPNsense be configured to properly handle state tracking when:

Initial SYN packets go directly from gateway (.80) to servers
Return traffic (SYN/ACK) goes through OPNsense (.85)

I understand this is not ideal, but disabling state tracking doesn't seem secure. Are there any settings for asymmetric routing that could help?

[dseven]

There's a state tracking mode called "sloppy", which https://man.freebsd.org/cgi/man.cgi?pf.conf(5) describes as:

"       sloppy
        Uses a sloppy TCP connection tracker that does not   check sequence
        numbers  at  all, which makes insertion and ICMP teardown attacks
        way easier.  This is intended to be used in situations where  one
        does  not   see  all  packets  of a   connection, e.g. in asymmetric
        routing situations.  Cannot be used  with   modulate  or  synproxy
        state."

TL;DR the conversation above, so not sure if it fits for you, but it is an option for rules in OPNsense.....

[Patrick M. Hausen]

No and no.

Only way would be to use a proxy in OPNsense like Caddy to terminate all incoming connections on OPNsense itself.

[pefen]

Thank you all for your advice. I'm a bit confused, I thought my scenario wasn't unusual and wouldn't require complicated solutions, but I guess I'm wrong.