Problems disabling NAT on WAN Interface (preventing double NAT)

[Reboot_Alex]

I discovered this issue while trying to diagnose and fix an NFS connection problems.


I am trying to segment my home network and am implementing Opnsense behind my ISP router/firewall (Edge router). I need to disable NAT on the Opnsense WAN interface. However, when I disable NAT, I am unable to access any system on the edge router network from within the Opensense LAN network.

No sure if this is important, but I have using Proxmox to virtualize Opnsense on a system with 4 NICs, with the following configuration:

   
• NIC 1: Direct passthrough to Opnsense - WAN Interface
• NIC 2: Direct passthrough to Opnsense - LAN Interface
• NIC 3: Reserved for Proxmox VMs. Connected to switch and Opnsense LAN
• NIC 4: Proxmox Management Interface (192.168.1.4)

My network is structured as follows:
Edge Router (192.168.1.1)
¦
¦-> Proxmox (192.168.1.4)
¦-> DNS (PiHole) (192.168.1.5)
|-> Opnsense WAN (192.168.1.6)  <--------------> Opnsense LAN (10.0.99.1)
¦-> NAS (192.168.1.20)                                        ¦
¦-> Test systems #1 (192.168.1.49)                       ¦-> Test system #2 (10.0.99.10)

I have changed the following settings in Opnsense:
"Interfaces" > "WAN" => unchecked "block private networks" and "block bogon networks". Set static IPv4 address.

To test/diagnose the problem, I have only the following firewall rules implemented:
"Firewall " > "Rules" > "LAN"

   
• Allow ICMP pings from edge router (192.168.1.1) to WAN address (192.168.1.6)
• Allow ICMP pings from edge network (192.168.1.0/24) to WAN address (192.168.1.6)
• Allow ICMP pings from edge network (192.168.1.0/24) to LAN network (10.0.99.0/24)
• Allow allow protocol/ports from edge network (192.168.1.0/24) to the LAN network (10.0.99.0/24)
• Allow the edge network (192.168.1.0/24) to access the Opensense WebUI (192.168.1.6) on port 80;443

"Firewall " > "Rules" > "LAN"

   
• Allow access to DNS server (192.168.1.5)
• Allow access to Opnsense WebUI (10.0.99.1) on the LAN network (10.0.99.0/24)
• Allow ICMP Pings from LAN Network (10.0.99.0/24) to the WAN network (192.168.1.0/24)
• Allow any system (*) to access the WAN Network (192.168.1.0/24)
• Allow access to the Internet, but block other private networks (!NonPublicIPs)

I added a static route on the edge router, to route LAN Addresses (10.0.99.0/24) via the Opnsense WAN interface (192.168.1.6)


To trying and isolate the problem, i tested the following:

1) Firewall disabled:

   
• LAN network can ping and access systems on the router/WAN network, and vice versa
• Conclusion: the routing is working (static route on the edge router)

2) Firewall enabled, NAT Enabled:

   
• LAN network can ping and access all systems on the WAN
• WAN network can ping and access the LAN network
• Cannot access the Opnsense WebUI on the WAN interface (192.168.1.6) from the wan network
• Cannot access NFS on NAS (192.168.1.20) from LAN Network (10.0.99.1) - Access has been granted on the NAS
• Conclusion: The firewall rules work (partially. WebUI?!). NAT is breaking the NFS access

3) Firewall enabled, NAT disabled

   
• LAN network can access the Internet
• LAN network cannot ping or access WAN Network; connections timeout. Firewall logs are green and show that the traffic passed through the LAN and WAN interface. Firewall logs do not show the return traffic being blocked
• Conclusion: Unsure. Perhaps the traffic is being miss-routed or blocked somewhere

Question:

• How to properly allow access between WAN and LAN without NAT?
• What settings have I not configured correctly?

P.S. I am not interested in implementing a transparent bridge, I required separate subnets.

[Saarbremer]

Hi,

depending on your edge router we could investigate more. However, in this very setup you're doomed without NAT.
Create a transfer network and proper routing on the edge router.

[Reboot_Alex]

However, in this very setup you're doomed without NAT.

I do not understand why this setup is doomed without NAT. Everything behind the edge router (and including the edge router) is a private network. This is a normal setup for an internal firewall, and best practice in an corporate environment.

Here is an example that I found:


As shown in the image, I am trying to implement the internal firewall. In such a scenario there is no need to have NAT, for systems in the internal or EDMZ network to access systems in the DMZ network.

Create a transfer network and proper routing on the edge router

As I stated in the original post, when the OPNsense firewall is disabled, systems on both sides of OPNsense can access each other. This means that the routing is working.

The problem I am trying to resolve while removing double NAT.
When the firewall is enabled and NAT is disabled, systems on the WAN side of OPNsense can access the LAN side systems, but not the other way round (these are still all internal private systems on my network). This means that OPNsense is either blocking the traffic or mis-routing in this configuration.

What could be causing this issue?

[Maurice]

That's not a good way to segment a network. You shouldn't place any hosts in a subnet which is used to interconnect routers (there are exceptions of course).

You'd have to manually create static routes to the OPNsense LAN on all of the hosts in this subnet, not just on the ISP router. Otherwise, when connecting to these hosts from the OPNsense LAN, OPNsense will send the request directly to the host, but the host will send the reply to the ISP router, not to OPNsense. That's the only route these hosts know - the default route. This can lead to all kinds of issues because of asymmetric routing.

Instead, create multiple LANs on OPNsense and put all your hosts in these LANs. Or create a second LAN on the ISP router which is only used as a link to OPNsense.

Cheers
Maurice

[Saarbremer]

In your setup you have two choices:

* Create a transfer network segment, separate IP range, only routers. Define other networks behind routers as needed and configure routing (as far as needed). That's exactly what your diagram says.

* Put OPNsense in the same network segement as your existing hosts ("edge router network") and force OPNsense to play masquerade act as a single host. This is usually achieved by NAT.

Remarks to the Web GUI:
*Unless you explicitly allow HTTP(S) on WAN to your OPNsense's IP, you won't be able to access it from the outside. You rules seem to provide this, but they obviously don't.

*Make sure, listening interfaces for the web gui include WAN.

*Watch out for possible security risk when you accidentally or purposely connect OPNSense directly to the internet.