Access in OPNsense issues with a AlmaLinux server that has two interfaces

[Cognoquest]

OPNsense 22.7.11_1-amd64
FreeBSD 13.1-RELEASE-p5
ESXi 7.0 Update 3
AlmaLinux 8.7

Hello All,

I am attempting to access a mail AlmaLinux 8 Server in the OPNsense DMZ zone from the LAN zone. The mail AlmaLinux 8 server was built with two interfaces to provide both public/WAN and private/LAN(private) accesses.

My OPNsense router configuration includes multiple zones/interfaces including WAN, LAN & DMZ zones. Obviously I have completely open all the involved firewall zones for solving this issue. I even created another AlmaLinux server(single interface) confirming access crossing from the LAN to the DMZ zones for the OPNsense router.

If the client accessing the mail server sits on the same subnet and zone as the mail LAN(private) interface, I have no access issues. But as soon as I move the mail server LAN(private) interface to the DMZ zone. I lose all LAN(private) accesses to the mail server. Hence: Traceroute from the OPNsense router, the mail server is found. Traceroute from a client in OPNsense LAN zone, the mail server is not found, the access stops at the OPNsense interface.

Thank you for reading this post.

P.

[Cognoquest]

After investigation I have discovered that this is not an OPNsense issue. Not really surprising since this was only happening for the AlmaLinux servers that has two interfaces. I did not grasp that the AlmaLinux setup requires two default interface when moved to the DMZ, often referred as (hot) potato routing, or deflection routing. Though it currently works on the LAN interface, not sure why?

The modification to the AlmaLinux servers requires me to configure the routes to return the packets on the same route they arrived. One of the issues that I will face is that the WAN route is configured via PPPoE, there is currently a bug with the Firewalld service reload command not working with PPPoE complicating things: https://github.com/firewalld/firewalld/issues/878.

Perhaps there is an alternative approach with OPNsense for Running the AlmaLinux interface via the DMZ interface. I am open to suggestions. Thank you for reading this post.

Philippe