issue with setting different gateway on target server with inbound traffic

[OhmegaStar]

Hi,

I have an issue in my network after adding opnsense 21.7 firewall between router and network.
I'm usure of the cause of the issue, but know that the issue started after moving the router behind a opnsense hw firewall (dell server)
I must apologize in advance if my explanation is not totally on point, I'm very much unsure of cause of the issue.

My network setup and topology is like this:

Network:
Windows AD Domain Joined Network 192.168.0.0/24
DNS: Windows AD > Dreammachine Pro > opnsense > internet

Servers: Mix of Windows and Linux (ubuntu) servers.

Internet:

Main Gateway / Router / Internet connection
4G WAN Bridged Router (unlimited traffic / no data caps) to Ubiquity Dreammachine Pro ip 192.168.0.1
Serves as Main Gateway for all DHCP clients
   Gateway 192.168.0.1
   
Internet works just fine on all clients and servers in the network through the primary gateway


Secondary Gateway / Router / Internet connection
4G WAN Bridged Router (Static IP, Inbound traffic enabled, Limited traffic / Data capped) to opnSense 21.7 Ip 192.168.0.10
Serves as Inbound traffic gateway (self hosted websites, mail etc.).
Inbound web traffic arriving at the secondary router is routed in opnsense (port mapped) to the appropriate target server in the internal network.
However after putting the secondary router behind the opnsense, the target server needs to have the opnsense ip as the gateway address for the inbound traffic "to work" - ie. webpages do not respond on the outside, smtp connections time out if not set.

Before moving behind opnsense the setup was somewhat similar, with the secondary router not in bridge mode but with port mapping routes in the router similar to the routes made in opnsense. There i did not need to set the secondary router as gateway for inbound traffic to work.


Setting the secondary gateway on any server causes any other (internally generated) internet traffic from that server to pass out through the secondary gateway, causing the network connection to reach datacaps - this is obviously not desired.

How do i find the cause of this issue in (I assume) my opnsense configuration?

Br,

Henrik

[OhmegaStar]

This issue is resolved by setting up hybrid outbound NAT, with manual rules for my internal network LAN and the specific port targetted so the responses from the server is routed back out of the firewall.

/OhmegaStar