Can't enable seemingly simple port forwarding scenario

[jttocinog]

I'm trying to learn OPNsense using a virtual environment I have set up for testing at home, using VirtualBox. The setup is as follows.

I have my home network, with a home router that offers DHCP for the 192.168.50.0/24 network. The gateway is at 192.168.50.1.

My main computer receives its ip address via DHCP. Let's say it gets assigned the 192.168.50.100 address.

Then I have a VM in VirtualBox with OPNsense freshly installed. It's got two network adapters, the first one (em0, which would be the WAN adapter) in bridged mode, which receives an IP address from the home network (let's say it gets the 192.168.50.150). The other adapter (em1, configured as the LAN adapter) is in internal network mode. The internal, local network uses the 10.10.0.0/24 addressing, and OPNsense uses the 10.10.0.1 address within that network.

Finally I have a Ubuntu machine with just one adapter in internal network mode. It's got the 10.10.0.30 address (received via DHCP from OPNsense). It's got a web server installed, listening on port 80. That machine has proper internet access using OPNsense as the gateway.

What I'm trying to achieve is a simple port forwarding of that web server outside the local network. In OPNsense, I go to Firewall > NAT > Port Forward, set the destination to the WAN address, the destination port to HTTP, the redirect target IP to 10.10.0.30 and the redirect target port to HTTP.

If I now try to access the port 80 of the OPNsense machine from my host machine, that is, connecting from 192.168.50.100 to 192.168.50.150:80, I can see that the internal machine receives the HTTP GET request, but I get an error in the host machine and the website never loads.

Most of the time I get a "connection reset" error. I think my home router is somehow interferring, because if I run wireshark in my host I can see the router responding instead of the OPNsense machine.

I can't make sense of it.

[newsense]

Virtualbox settings for OPNsense VM - on the bridged interface you'll have to allow Promiscuous Mode

[dseven]

Firewall > Settings > Advanced > Disable reply-to

[jttocinog]

Firewall > Settings > Advanced > Disable reply-to

OMG this did it! Thanks a lot! It would be great to know where I can read more about this. In fact I had never heard of the term "Multi-WAN". Thanks again.

[dseven]

This comes up fairly often. If you search the forums for "reply-to", you'll probably find some examples. Basically the reply-to is set on port forwards by default to try to ensure that the correct return path is used when there are multiple WAN interfaces active simultaneously. https://docs.opnsense.org/manual/how-tos/multiwan.html mentions it, but doesn't really explain it.

[EricPerl]

It's biting a lot of folks that use OPN on a private network (like the OP, and me for my test instance).
Couple days ago, it resolved another thread reporting poor iperf throughput in one direction only (at some point, that user even connecting the client to WAN directly and the perf issues persisted! No issues with pfSense either).

It still looks like a weird default to me (justification is historical reasons).
It would be quite easy to include a step in the multi-WAN how-to (in fact only in the load balancing case, on top of the bridging caveat).

It's so undiscoverable in the private network deployment case...
I saw the side effects (traffic hitting the gateway of the production VLAN used by the WAN interface of my test instance).
I still had no clue why until @dseven pointed me to that setting.

It doesn't help that the documentation for the setting leads users to believe it only/mostly applies to multi-WAN.
Only the warning explains the end result, when it is read carefully.

[dseven]

I agree, and have been thinking the same thing. I imagine it's too late to flip the logic (the option really should be something like "force reply-to", and should be off by default), as that would break existing deployments, but maybe there's a case for the "disable reply-to" option to be on by default for new installations, and the Multi-WAN instructions updated to recommend turning it off if needed.

[jttocinog]

Maybe they could add an additional step in the initial configuratio wizard for this setting? Seems important enough I think. Or maybe not a direct question about this setting but rather about the environment where OPNsense is being deployed. Surely, just like the aforementioned one, there are more settings that should be different in a private-network-within-private-network environment, and a simple "What environment are you deploying OPNsense in?" question could set a couple of defaults just for ease of mind.