[SOLVED] Double NAT, gateways and internet access

[mogster]

I'll preface this by saying while I'm pretty good with IT in general (I'm a senior ICT technician), networks aren't my thing and this question will probably sound really daft, but here goes.

I've just set up a firewall running OPNsense, mainly to wall off a web server from the rest of my network. This is behind my ISP router with a double NAT, which I know isn't ideal but I don't really want to interfere with the rest of my network for the sake of my wife! This is basically working now, and I can access all my stuff over the internet using the Caddy reverse proxy plugin.

Where things get messy is when I try to access sites from my ISP router network using the OPNsense WAN interface, and I've narrowed it down the gateway. If I set the WAN interface gateway rule to my ISP router, I can get out to the internet from LAN clients but I can't access my LAN websites using NAT rules from my WAN network. If I set the gateway rule to "disabled", my NAT rules all spring into life and I can access my websites from my WAN, but I can no longer access the internet from my LAN.

I'm sure there must be a simple solution to this, but I seem to be hitting a wall. Does anyone have any advice?

EDIT: Solved!

After a lot of trial and error I remembered that I had to set an outbound NAT rule for my VPN in order to access addresses on my internal WAN, so I thought I'd try the same for my LAN. Just like that, I can access WAN addresses from my LAN, and more importantly break out to the internet.

I also figured out how to stop the little tune my firewall plays when you power off and on, so that's a double achievement.

Thanks to those that offered suggestions, and sorry for my no doubt very confusing post.

[bartjsmit]

What about a bridge firewall? https://docs.opnsense.org/manual/how-tos/transparent_bridge.html

[mogster]

What about a bridge firewall? https://docs.opnsense.org/manual/how-tos/transparent_bridge.html

That's certainly an option. I actually did exactly that when I first ran into the issue and it worked well, but I wanted to have another crack at setting it up this way.

Honestly though the bridge setup is probably more sensible for my network.

[mogster]

Well, I have a workaround. I was going to set up WireGuard on the firewall anyway, it turns out this works just as well inside my network as it does over the internet. This way I can just connect over the VPN if I need to get to the Firewall's LAN network, but still leave the WAN gateway set.

Let's not speak about how long I just spent trying to troubleshoot this VPN before I realised I had to turn it off and on again before new peers would work. Some IT technician I am.

[FLguy]

Sorry mogster, I'm not 100% sure if I'm tracking exactly what you're trying to do. But I would suggest looking at NAT reflections. 

The other suggestion is Disabling reply-to on WAN rules (Firewall > Settings > Advanced). 

[mogster]

Solved!

After a lot of trial and error I remembered that I had to set an outbound NAT rule for my VPN in order to access addresses on my internal WAN, so I thought I'd try the same for my LAN. Just like that, I can access WAN addresses from my LAN, and more importantly break out to the internet.

I also figured out how to stop the little tune my firewall plays when you power off and on, so that's a double achievement.

Thanks to those that offered suggestions, and sorry for my no doubt very confusing post.