[SOLVED] Trouble with Return Traffic Routing through LAN to DMZ

[kaisertux]

Hi Everyone,

I am currently experiencing an issue in my network setup where return traffic from a host in the LAN network isn't routing correctly back to a host in the DMZ. Below is a brief overview of my network configuration and the issue:

Network Setup:
External Firewall (OPNsense): Access to the internet
Internal Firewall (OPNsense): Configured with three interfaces:
DMZ (WAN): 192.168.2.2/24 (Accepts private networks, DefaultGateway: 192.168.2.1 (External FW))
LAN: 192.168.3.1/24 (NAT disabled)
MGT (OPT1): 192.168.4.20/24 (Access for the webfrontend)
Hosts:
LAN: 192.168.3.2 (Can access both the internet and 192.168.2.12 in DMZ)
DMZ: 192.168.2.12 (Struggling with return traffic to 192.168.3.7 in LAN)
LAN: 192.168.3.7 (Target host for 192.168.2.12)
Routing on 192.168.2.12: Network 192.168.3.0/24 behind 192.168.2.2
Issue:
When the DMZ host (192.168.2.12) tries to access the LAN host (192.168.3.7), the outgoing traffic traverses the internal firewall correctly, reaching 192.168.3.7. However, the return traffic from 192.168.3.7 reaches the internal firewall but is then sent incorrectly to the external firewall (192.168.2.1) rather than back to 192.168.2.12.

Questions:
Has anyone experienced a similar issue with return traffic being incorrectly sent to the default gateway instead of the initiating host within the DMZ?
Is there a specific configuration or rule within OPNsense that might be redirecting this return traffic incorrectly?
Any recommendations on additional debugging steps or configurations to check?
Any insights or suggestions on why the internal firewall may be routing this return traffic to the external firewall, and how to resolve it, would be greatly appreciated.

Thank you so much in advance!

Regards,
Richi

[Monviech (Cedrik)]

You could start with some packet captures on the opnsense.

Go into the ssh shell and there select the console. Its best to open 2 ssh shells because then you can see where the packets enter and leave the interface.

With "ifconfig" you can see all your interfaces.

In both consoles type (replace lan with your device name):
tcpdump -i lan proto icmp -n
tcpdump -i dmz proto icmp -n

Afterwards start a ping and look what happens to the packets.

[Maurice]

Try explicitly disabling reply-to in the advanced options of the firewall rule which allows inbound connections on the WAN interface.

Cheers
Maurice

[Monviech (Cedrik)]

I think the problem is that the setup is:
Opnsense 1: 192.168.2.1
Opnsense 2: 192.168.2.2
Client: 192.168.2.12

The default gateway of the dmz client and opnsense 2 is probably 192.168.2.1. So it always sends the packets there.

The easiest way to fix this problem is not putting clients into the transfer net between opnsense 1 and opnsense 2. The DMZ client should be in its own seperate subnet.

[kaisertux]

You could start with some packet captures on the opnsense.

Go into the ssh shell and there select the console. Its best to open 2 ssh shells because then you can see where the packets enter and leave the interface.

With "ifconfig" you can see all your interfaces.

In both consoles type (replace lan with your device name):
tcpdump -i lan proto icmp -n
tcpdump -i dmz proto icmp -n

Afterwards start a ping and look what happens to the packets.

That's exactly what I did. That is why I know already exactly where the traffic goes. But honestly I did not run the tcpdump on the console, since there is such a nice UI to do that in opnsense

[kaisertux]

I think the problem is that the setup is:
Opnsense 1: 192.168.2.1
Opnsense 2: 192.168.2.2
Client: 192.168.2.12

The default gateway of the dmz client and opnsense 2 is probably 192.168.2.1. So it always sends the packets there.

The easiest way to fix this problem is not putting clients into the transfer net between opnsense 1 and opnsense 2. The DMZ client should be in its own seperate subnet.

True, the easiest way to solve this issue is, just to avoid the issue. But that is no solution. For clarification, my setup has the architecture as you can see in wikipedia:
https://en.wikipedia.org/wiki/DMZ_(computing)#Dual_firewall
This may look oversized, but in fact it's quite common, expecially if the external firewall is managed by an external company, while the internal firewall is managed by the IT of the company itself. In my case, it's a testing environement, where I want to check, if opensense could be a solution for a project.

[kaisertux]

Try explicitly disabling reply-to in the advanced options of the firewall rule which allows inbound connections on the WAN interface.

Cheers
Maurice

I will check that, that's pointing into the right direction I guess

[kaisertux]

Try explicitly disabling reply-to in the advanced options of the firewall rule which allows inbound connections on the WAN interface.

Cheers
Maurice

Yeeehh! Many thanks, that was working. Now I will try to figure out, what this option exactly means.

[Monviech (Cedrik)]

Sorry for being wrong. I learned something too from this.

Disabling "reply-to" stops the firewall from forcing return traffic through the default gateway of the interface, allowing the firewall to use its routing table to determine the correct path for return traffic, thereby resolving asymmetric routing issues.

[kaisertux]

Sorry for being wrong. I learned something too from this.

Disabling "reply-to" stops the firewall from forcing return traffic through the default gateway of the interface, allowing the firewall to use its routing table to determine the correct path for return traffic, thereby resolving asymmetric routing issues.

Ahh! Thanks, that makes sense. I thought, why the hell is this firewall not following the routing table