double nat design considerations?

[4fred]

So today at home I have one network running on the router supplied by my ISP. The only thing I have done is add a pi hole dns and that's it. The family runs lots of stuff on the network and I kind of do not trust it. Vacuum cleaner, phones and tablets, streaming boxes TV's and all kinds of trinkets. And if you EVER do something to make that network go down, well I guess you know what happens...
So my thought is to make another network inside this one where I run my stuff and can play how ever much I like. Maybe gradually move devices (that I trust) to my network and leave ISP's network just for streaming devices and things I do not trust. If I'm sucessful maybe also replace the pi-hole with opnsense's unbound DNS. So I would have access from the new network to the internet trough my ISP's supplied network and also maybe grant some access to some things on "my" network.

So I would run a opnsense and behind that connect my devices to the newly built network. I've read up some on double nat and from what I understand it would work as long as you do not "Block private networks". Are there any more considerations I should be aware of?

The network supplied by my isp's router is 192.168.1.0/24 and I'd prefer not to change that.
So I was thinking of attaching WAN to this network and a new LAN inside of it, maybe 192.168.3.0/24?
Is this a good or a bad idea... ?
Not a network engineer but I'm trying to learn.

[passeri]

Are you providing services you can access from the internet? If not, double-nat is not a consideration. You may need single NAT depending on what internal services you offer or access, but most access tends to be outward.

Double NAT adds latency and possible extra maintenance work. In a fairly stable home network with current hardware this is no big deal.

[netnut]

Have you checked (and double checked) if you can configure a static route in your CPE. A FritzBox for instance allows you to add such route, pointing a static route from your CPE towards your OPNsense box will eliminate any NAT requirements on OPNsense and so "Double NAT".

If your ISP locked down the complete CPE there isn't much choice unfortunately...

[4fred]

I do not publish anything to the internet from my home. If the need comes up I can go with a cloudflare application tunnel or something creative like that.

I can somewhat change network stuff but not a lot DHCP range and DNS server and just some very basic things.

So setting up an opnsense with wan/lan leaving wan on dhcp and setting lan to 192.168.3.0/24 would work just fine?

[netnut]

So setting up an opnsense with wan/lan leaving wan on dhcp and setting lan to 192.168.3.0/24 would work just fine?

Yes, just to be sure to UNCHECK the Block Private Networks on the WAN interface (which you normally leave checked).

[4fred]

Thanks!
I'll give it a go and let's see if/how it works.

[4fred]

There is something funny.
I set up opnsense with wan/lan, did just some basic stuff and like dns/dhcp range and set lan ip to 192.168.3.1 and the dhcp network just followed automatically.
In settings changed the webport to 5858 and disabled redirection.
Behind the FW I set up two clients and everything worked as expected. Access stuff on the internetz and access to resources on the other LAN (192.168.1.0/24) worked as expected.

Now to the problem.
My plan is to move some stuff from the old lan (192.168.1.0/24) to the new lan (192.168.3.0/24), like my NAS and some other things. These things are used by devices that will still be on the old lan, TV's tablets and things. So to test that this would work I setup a webserver on one of the clients on the new lan and tried to open the port between the two and that does not work. If I test the webserver from the client that are on the same network it works but if I try from the old lan that must go trough opnsense that does not work.
Firewall/NAT/Port forward.
Interface: WAN
TCP Version: IPv4
Protocol: TCP
Source: (didn't change anything so any/any)
Destination: WAN Address
Destination port range from to: HTTP/HTTP
Redirect target IP: Single host or network, IP of the webserver: 192.168.3.101
Redirect target port: HTTP
Enabled logging and let it allow the filter rule by itself, also tested to manually create FW rule but no diffarance).
Basically just save and test, no dice.

I figured that this may be a problem with HTTP so I tried to allow RDP access trought the FW but that dont work.
I went in to logging live view and entered: dst contain: 192.168.3.101 up comes the green and nice things saying opnsense allowing the traffic and nothing is blocked, still it does not work.
So... routing?

Something is "funny" and I dont really know where to beging to look for what's wrong.
Anyone?

[passeri]

You speak of opening a port between local networks then create a port forwarding rule from the WAN?

For local access you need PASS a rule IN to the interface of the source network with destination target interface network.

As it is you are setting up external access. That is a separate thing.

[4fred]

I did the setup with wan/lan, so two nics.
One is considered WAN the other is LAN.
So how is this not considered allowing an external network when it's coming from the wan interface?

Please explain cuz I do not understand.
And could you please explain how to do it?

[passeri]

If you have a basic Opnsense setup, then you should now be able to access the actual WAN (through the ISP router) from devices on its LAN side. Have you tested this yet?

If you have a basic Opnsense setup, then you should not be able to access the devices on its LAN side from anything on the WAN side of Opnsense. Have you tested this yet?

If either test fails, then there is likely to be a problem, that you do not have a default Opnsense setup or else you have a block elsewhere.

It is not necessary, in fact undesirable, to switch off blocking of local networks on the WAN side of Opnsense if the whole idea is to block untrusted devices elsewhere under your ISP router.

Since I have no diagram, I am working on my understanding of your verbal description of your layout and intended results.

[4fred]

192.168.1.10 --> 192.168.3.101 on port 80 does not work.
192.168.3.101 --> 192.168.1.10 on port 80 do work.


From my ISP I got a router. This router is mostly locked for me and I cannot do much with it. It gets a DHCP address on its WAN and it shares out DHCP addresses on its LAN. Family uses this and connects EVERYTHING to it. If I touch this network and breaks it my life is over and done, the rest of the house would turn me into pancakes dough.
Let's call this ISP-LAN and it uses 192.168.1.0/24 and all addresses is handed-out by ISP router and it sits on 192.168.1.1.

Onto opnsense.
I take a box and attach a switch to it's LAN port. I attach two PCs to it. On the box I install opnsense using yes-next-finish type installation. WAN is NOT! connected during install. I then change opnsense LAN IP to 192.168.3.1 and reboot it. Now the two connected PCs gets DHCP addresses from opnsense and they are on the 192.168.3.0/24 network. I UNCHECK the Block Private Networks on the WAN interface, all good everything works but not connection anywhere. I can from the two PCs talk to opnsense and they can talk to it, nice.
Now I connect the opnsensebox WAN port to the ISPs router on the LAN port, and opnsense now get's a WAN-IP on the 192.168.0.1/24 network and have internet access, the two connected PCs also get internet access, all good.

Now the problem:
192.168.3.101 --> 192.168.1.10 on port 80 do work.
192.168.1.10 --> 192.168.3.101 on port 80 does not work.
FROM the ISP-LAN I cannot access anything on the opnsense LAN, 192.168.3.0/24.
FROM the opnsense LAN, 192.168.3.0/24 I can access everything on the ISP-LAN, 192.168.1.0/24.

Let's say I wanna move a webserver from ISP-LAN 192.168.1.0/24 to the opnsense LAN 192.168.3.0/24 - what do I have to do?

[passeri]

192.168.3.101 --> 192.168.1.10 on port 80 do work.
192.168.1.10 --> 192.168.3.101 on port 80 does not work.
FROM the ISP-LAN I cannot access anything on the opnsense LAN, 192.168.3.0/24.
FROM the opnsense LAN, 192.168.3.0/24 I can access everything on the ISP-LAN, 192.168.1.0/24.

You called it a problem? The above is exactly expected behaviour. WAN side of Opnsense (192.168.1.x) blocks incoming traffic but anything can get out from behind it (192.168.3.x). Your work to this point is complete, or else your aim was not as has been understood.

Now, the question:

Let's say I wanna move a webserver from ISP-LAN 192.168.1.0/24 to the opnsense LAN 192.168.3.0/24 - what do I have to do?

Assuming a standard web server, these steps:
1. Install the web server and ensure it has a stable IP address in the 192.168.3.x range.
2. Add to Opnsense a Firewall NAT rule for port 80 IN to the WAN address, redirect to port 80 at the web server's IP address.
3. Test.

As a comment, if your aim is to protect family devices from experimental devices, then the proper place for your internal firewall is in front of the family devices, not the lab/server. What you are currently doing is testing to avoid later pancake risk, not protecting important things yet. That would also simplify things in that your configuration could stop where it is, no NAT needed (nor allowing local networks).

[4fred]

"Yes I called it a problem" ...
In #6 in this thread I described what I had done to try to open the port just like you describe for me how to do and it does not work. I know that opnsense by default blocks everything from WAN, that is why I'm setting it up.

Step one is to protecte/remove the things -I- care about protecting from the family's stuff. Not just buy anything and walk over to the router and connect to network. Cheap trinkets made in wherever probably crap leaking god knows what... THESE are the "experimental devices".

[passeri]

Would you mind showing a screenshot of your NAT rule, rather than a description?

[4fred]

Here ya go