Messages

I've already updated to the latest OPNsense version, but the equal misbehavior as previously described remains

Version: OPNsense 22.1-amd64

Can someone help me?

[Issue:]

I using a virtual environment (esxi) with several hosted virtual clients and servers for testing purpose and cannot access from OPNsense WAN the hosted services (http, rdp, etc.) at OPNsense LAN.

The virtual environment is segregated from my private home network (PHN) with an OPNsene firewall. The OPNsene WAN interface is connected to my private network and should use my internet gateway to connect to the internet. I am using my test environment since about 2 years and had no issue with pfSense the gateway and NAT worked great. About 4 months ago, I decided to switch to OPNsene and my issues started there.

Due to the virtualization, I can simply change the firewalls (OPNsense/pfSense), so I can probably exclude that the issue is at my infrastructure or network configuration. NAT is used at OPNsene to access hosted services in the virtual environment and the gateway should be used so that the server and clients in the virtual environment (LAN 10.0.20/24) can access the Internet. 

Network Infrastructure:
Internet <-> LTE Bridge <-> (DHCP WAN) Router (LAN GW 10.0.10.1) <-> PHN (LAN 10.0.10.0/24) <-> (WAN 10.0.10.200) OPNsene (LAN 10.0.20/24)

Issue:
I cannot access the services in virtual environment (10.0.20.0/24) from private home network (LAN 10.0.10.0/24) when I use OPNsense.

Observations/Analyze with Wireshark
NAT and Firewall rules should be ok, because the services behind the OPNsense Firewall (LAN 10.0.20.0/24) receives the requests  from PHN (LAN 10.0.10.0/24) and sends responses. The responses are also at OPNsense WAN interface but the destination MAC is not equal with the origin request MAC. The MAC address is changed by OPNsense to the MAC Adress from my Internet GW (10.0.10.1). So return packages were not sent to my requesting PC from (10.0.10.0/24) but sent to the Gateway MAC (10.0.10.1).

When I disable the Gateway functionality at OPNsense (System/Gateways/Single) the destination MAC Adress is not changed by OPNsense and the origin requester receives the response. If Gateway is disabled NAT works great and all services from OPNsense LAN (10.0.20.0/24) are reachable from 10.0.10.0/24 and can be used.
IF Gateway is enabled, responses are forwarded to my Internet Gateway (10.0.10.1)

Network IPs
Internet Gateway private network (IGW): 10.0.10.1
Private home network (PHN): 10.0.10.0/24
OPNsense WAN: DHCP (WAN IP: 10.0.10.200, Gateway: 10.0.10.1, DNS: 10.0.10.1)
OPNsense LAN: DHCP 10.0.20.0/24

Extract from: System/Routes/Status
Proto  Destination     Gateway     Flags  Use       MTU   Netif   Netif
ipv4    default            10.0.10.1    UGS    805      1500   vmx1   wan
ipv4    10.0.10.0/24   link#2        U        81397   1500   vmx1   wan
ipv4    10.0.10.200    link#2          UHS   0       16384   lo0   Loopback
ipv4    10.0.20.0/24   link#1          U   50687  1500   vmx0   lan

Extract from: System/Gateways/Single
Name                         Interface  Protocol    Priority    Gateway    Monitor IP    RTT    RTTd    Loss    Status    
WAN_DHCP (active)   WAN        IPv4          254       10.0.10.1    10.0.10.1    0.6 ms    0.2 ms    0.0 %    Online    Interface WAN_DHCP Gateway

WAN Interface: Block private networks – is disabled
Version: OPNsense 21.7.7-amd64

Any idea?