[Tutorial] - NAT Reflection/Hairpinning with OPNsense

[Monviech (Cedrik)]

LATEST VERSION: https://docs.opnsense.org/manual/how-tos/nat_reflection.html

The version below won't be updated anymore.

This is version 1.2 of a NAT reflection tutorial, being the result of this github threat: https://github.com/opnsense/core/issues/6650
I hope it helps making NAT reflection with the OPNsense a bit clearer.

Please read this warning and use manual rules as described below:
https://docs.opnsense.org/manual/firewall_settings.html#network-address-translation

Changelog:

• 19.07.2023 - 1.0 Initial version with manual NAT reflection/hairpin tutorial
• 20.07.2023 - 1.1 Expanded NAT and NAT Reflection explanations, corrected some formatting. Expanded on options for automatic NAT reflection.
• 21.07.2023 - 1.2 Made SNAT rule more restrictive to only hairpin to a single host as destination, not to the whole network. Changed some formatting and cleared some writing errors. Put some additional troubleshooting steps in.

0.1 What is NAT?

Because there aren't enough free IPv4 addresses, people invented a thing called "NAT" (Network Address Translation). It basically makes a router like the OPNsense translate IPv4 addresses to other IPv4 addresses.

For example, the external IPv4 address 203.0.113.1 is translated to the internal IPv4 address 172.16.1.1, and the other way around! Now things like "NAT Overload (Masquerading - SNAT)" are possible, allowing multiple internal IPv4 addresses to share a single external IPv4 address. But in return, you need "Port Forwarding (Destination NAT - DNAT) to make ports of an internal client accessable by external clients.

SNAT = Source Network Address Translation = Changes the source IP of a packet
DNAT = Destination Network Address Translation = Changes the destination IP of a packet
PAT = Port Address Translation = Changes the destination port of a packet

In summary, NAT saves IPv4 addresses, by enabling many devices to share the same internal IPv4 address spaces (RFC 1918, 192.168.0.0/16 - 172.16.0.0/12 - 10.0.0.0/8), and providing them only one external IPv4 address (or in some cases a few more).

If you create a Destination NAT (DNAT) rule, also known as "Port Forwarding", you give clients in the WAN the ability to access ports of the internal IPv4 address 172.16.1.1, by letting the OPNsense translate the destination from the external IPv4 address 203.0.113.1 to your internal IPv4 address 172.16.1.1. The OPNsense acts like a translator, translating IP addresses between client and server. The OPNsense writes all translations into a file, which is the NAT table. So it knows exactly how traffic should flow back and forth with the translations in place.

0.2 Why do I need NAT reflection?

Your own internal clients cant reach 203.0.113.1 with a normal port forwarding rule. The OPNsense receives the packet, checks the destination IP, checks the interface the packet is received, and drops it. That's because there is no rule in place, that tells the OPNsense where to send the packet to, and to allow it. For the OPNsense, it looks like itself should be receiving the packet, because the external IPv4 address is on its own WAN interface.

That's where NAT Reflection/Hairpin comes in play (as opposed to Split DNS which should be avoided if possible). It helps your internal clients to communicate with 203.0.113.1, by creating rules that use the OPNsense as the "translator" to the actual destination 172.16.1.1.

0.3 In my experience, there are three main combinations to do NAT reflection in the OPNsense.
Choosing ONE of those options and sticking to it will make things work as expected:

• Creating manual Port-Forward NAT (DNAT), manual Outbound NAT (SNAT), and automatic firewall rules
• Creating automatic Port-Forward NAT (DNAT), manual Outbound NAT (SNAT), and manual firewall rules
• Creating automatic Port-Forward NAT (DNAT), automatic Outbound NAT (SNAT), and manual firewall rules

0.4 Important: The firewall rules are NEVER created automatically when ticking the checkboxes for automatic NAT reflection (Firewall: Settings: Advanced). That is indeed by design. https://github.com/opnsense/core/issues/6650#issuecomment-1632016179

0.5 There are two different types of NAT reflection:
https://github.com/opnsense/core/issues/6650#issuecomment-1625135426

• Nat Reflection: The client and the server are in different subnets (layer 2 broadcast domains) and the OPNsense routes traffic between them. They CAN'T communicate directly by resolving ARP requests. You only need DNAT.
• Nat Hairpinning: The client and the server are in the same subnet (layer 2 broadcast domain). They CAN communicate directly with each other by resolving ARP requests. You need SNAT and DNAT.

0.6 My personal opinion and my best practice:

In my opinion, the best way to do NAT reflection in the OPNsense is not to use the automatic rules in (Firewall: Settings: Advanced). Creating the NAT rules manually works great, prevents mistakes, and reduces the amount of rules in the background.

Reasons:

• Automatic NAT reflection will create lots of unnecessary rules in the background. You can check them in the shell with "pfctl -s nat" and in "/tmp/rules.debug". Manual rules will tailor the OPNsense exactly to your NAT reflection needs.
• Manual NAT reflection WILL automatically create the necessary firewall rules in "Rules: Firewall: Floating"
• Automatic NAT reflection will create more SNATs than needed, turning all NAT Reflection into Hairpinning. It will create many unnecessary Outbound NAT (SNAT) rules for all interfaces.
• Automatic NAT reflection rules aren't visible in the GUI.
• The automatic NAT reflection features wouldn't be there if it wouldn't already exist, because its convoluted.  https://github.com/opnsense/core/issues/6650#issuecomment-1632048777

Possible Limitations:

• Manual Rules won't create NAT rules for the loopback interfaces of the OPNsense. I don't know why automatic rules create them, they might be needed for some plugins...?

0.7 In my examples you always have three networks:

WAN - 203.0.113.0/24
Webserver External IP - 203.0.113.1
Opnsense - 203.0.113.254

DMZ - 172.16.1.0/24
Webserver Internal IP - 172.16.1.1
Opnsense - 172.16.1.254

LAN - 192.168.1.0/24
Internal Client - 192.168.1.1
Opnsense - 192.168.1.254


0.8 The goal is the following:
Access the Webserver 172.16.1.1 port 443 with it's external IP 203.0.113.1 from a client in WAN, LAN and DMZ.
You can use aliases for everything, but I keep this as simple as possible here.

CHOOSE ONLY ONE OPTION, THEY ARE EXCLUSIVE TO EACH OTHER:

1.0 OPTION 1 - Creating manual Port-Forward NAT (DNAT), manual Outbound NAT (SNAT), and automatic firewall rules

1.1 Firewall: NAT: Port Forward

• Create a new rule:
• Interface: Select all interfaces in which clients are that should access the webserver from its external IP. In this example select WAN/DMZ/LAN. This will create a linked Firewall Policy in "Firewall - Rules - Floating" which allows all NAT reflected traffic.
• Protocol: Select TCP, because Websevers on port 443 use TCP
• Source: Any, because you don't know where the traffic to the webserver originates from.
• Source port range: Any or 1024-65535, it depends on how restrictive you want the firewall to be. There shouldn't usually be any traffic from source port 1023 and below to destination port 443 of a webserver.
• Destination: 203.0.113.1, which is the external IP of the webserver in this example. If you have only 1 IP, you can also take the default alias "WAN address".
• Destination port range: 443, or the alias "HTTPS"
• Redirect target IP: 172.16.1.1, which is the Webservers Internal IP in the DMZ
• Redirect target port: 443, the same as the Destination port range
• Description: Make sure to add a description like "NAT Rule Webserver 443" because the linked Filter rule association will use that as the name, and the Floating Firewall rule will have it in the description.
• NAT reflection: Set on Disable. Because you won't need it with your manual rules in place.

In verbose, the DNAT rule can be read like this:

If a packet is received by the OPNsense on the interfaces WAN,DMZ,LAN with protocol TCP from the source ip ANY and the source port range 1024-65535 to destination
ip 203.0.113.1 and destination port 443 -> rewrite the destination ip to 172.16.1.1 and the destination port to 443.

The automatic linked floating firewall rule will allow traffic to the destination IP 172.16.1.1 because NAT rules match before Firewall rules. That means the firewall receives the packet and the NAT rule converts the destination from 203.0.113.1 to 172.16.1.1 first, before passing the packet to the firewall filter.

Now, the traffic from the Client 192.168.1.1 in the LAN, and the Client in the WAN can access the Webserver.
But somehow... the Webserver 172.16.1.1 in the DMZ can't reach it's own NATed external IP 203.0.113.1 and it's own HTTPS port 443. Thats where the next rule comes in:

1.2 Firewall: NAT: Outbound

• Select "Hybrid outbound NAT rule generation" That way you can have manual outbound rules but still profit from automatic Masquerading rules.
• Create a new rule:
• Interface: DMZ, which is the interface the Webserver is in.
• Protocol: TCP
• Source Address: DMZ net, which is the Alias for the Network 172.16.1.0/24
• Source Port: Any
• Destination Address: 172.16.1.1, which is the internal Address of the Webserver
• Destination Port: 443, or the alias HTTPS
• Translation/target: DMZ address, which is the Alias for the OPNsense Interface IP in the DMZ Network 172.16.1.254
• Description: Hairpin NAT Rule Webserver 443

Now the Webserver and all other Clients in DMZ can reach the webserver with it's external IP. This kind of NAT is called "Hairpin NAT" and a combination of DNAT and SNAT. You need this because otherwise the Webserver wouldnt communicate "Webserver <-> OPNsense <-> Webserver" but "Webserver -> Opnsense -> Webserver <-> Webserver", making any traffic asynchronous and HTTPS will fail to work.

In verbose, the SNAT rule can be read like this:

If a packet is received by the OPNsense on the interfaces DMZ with protocol TCP from the source net 172.16.1.0/24 and the source port ANY to destination ip 172.16.1.1 and destination port 443 -> rewrite the source ip to 172.16.1.254 and answer from the OPNsense firewall interface.

2.0 OPTION 2 - Creating automatic Port-Forward NAT (DNAT), manual Outbound NAT (SNAT), and manual firewall rules

2.1 Firewall: Settings: Advanced:

• Enable "Reflection for port forwards" to create automatic rules for all "Port Forwarding" rules in "Firewall: NAT: Port Forward" that have "WAN" as interface.

2.2 Firewall: NAT: Port Forward

• Create the NAT rule as in 1.0 OPTION 1 - 1.1 Firewall: NAT: Port Forward - But change the following things:
• Make sure that your "Port Forwarding" rule has ONLY "WAN" as interface.
• Leave "NAT Reflection" on "System Default"

2.3 Firewall: Rules: Floating

• Create a new rule:
• Action: Pass
• Interface: WAN,DMZ,LAN - All Interfaces that should be permitted to access the Webserver through NAT Reflection
• Protocol: TCP
• Source: Any
• Destination: 172.16.1.1 - The internal IPv4 address of the Webserver the rule matches for.
• Destination port range: 443, or the Alias HTTPS
• Description: DMZ Webserver NAT Reflection - Or something else.

2.4 Firewall: NAT: Outbound

• Create the NAT rule as in 1.0 OPTION 1 - 1.2 Firewall: NAT: Outbound

3.0 OPTION 3 - Creating automatic Port-Forward NAT (DNAT), automatic Outbound NAT (SNAT), and manual firewall rules

3.1 Firewall: Settings: Advanced:

• Enable "Reflection for port forwards" to create automatic rules for all "Port Forwarding" rules in "Firewall: NAT: Port Forward" that have "WAN" as interface.
• Enable "Automatic outbound NAT for Reflection" to create automatic SNAT rules for all "Port Forwarding" rules in "Firewall: NAT: Port Forward" that have "WAN" as interface. The firewall will now answer with its OWN IP on each interface in response to NAT Reflected traffic.

3.2 Firewall: NAT: Port Forward

• Create the NAT rule as in 2.0 OPTION 2 - 2.2 Firewall: NAT: Port Forward
  

3.3 Firewall: Rules: Floating

• Create the floating firewall rule as in 2.0 OPTION 2 - 2.3 Firewall: Rules: Floating
  

4.0 Troubleshooting NAT Rules

• Open SSH shell:
  Shows all NAT rules active currently in the system:
  pfctl -s nat
  "rdr" means redirection Firewall: NAT: Port Forward rules
  "NAT" means Firewall: NAT: Outbound rules
• Shows all NAT rules in the OPNsense debug:
  cat /tmp/rules.debug | grep -i nat
• Look at the default drops of the firewall live log in "Firewall: Log Files: Live View".
• Turn on logging of the NAT and Firewall rules you have created, and check if they match in "Firewall: Log Files: Live View". NAT rules have the label "NAT" and firewall rules have the label "Description you gave your rule"
• In "Firewall: Diagnostics: Sessions" you can check if there is a session between your internal client and your internal server, and which rule matches to it. You're looking for "STATE ESTABLISHED".
• Use tcpdump on the client, the opnsense and the server, and test if the traffic goes back and forth between the devices without any mistakes. Look for TCP SYN and SYN ACKs. If there are only SYN then the connection isn't established and there are mistakes in your rules.

Feedback is welcome if aspects of the tutorial can be improved, or if there are mistakes.

[lilsense]

So now that the new version is out, is this contiuing to be the same? I.e. is this the nature of the beastie (FreeBSD).

[Monviech (Cedrik)]

I'm not sure if I understand your question the right way.
- How this kind of NAT works is not exclusive to FreeBSD.
- How the different options in the GUI of the Opnsense influence which rules are generated in the underlying OS is whats explained in this tutorial. And if there arent any commits that change this behavior, this tutorial stays valid.

Maybe I'll work on including this tutorial into the official docs, in a less opinionated way of course.

[cookiemonster]

this is very informative, thanks.

[Monviech (Cedrik)]

this is very informative, thanks.

Thank you very much. If you find anything you would add or improve let me know. Just today it was merged into the Opnsense docs.