[solved]Issue regarding dedicated reverse proxy and lan access (from lan only)

[touch1337]

Hello,

Sorry about the title, I have no idea how to phrase my issue properly. I'm new on OPNsense and I'm trying to add it to  my current, very basic, setup.

My issue is very simple and I'm sure very easy to fix, but I couldn't find the option nor formulate properly the search keyword to find any related topic (I'm a noob, don't know the exact words for all the things yet ^^').

--

I have multiple virtual machine on a Proxmox host. I have a NGINX reverse proxy to deal with the internal port forwarding and the https certificate). It's working like a charm without OPNsense, of course.

When I add OPNsense and setup the port forwarding rule for my reverse proxy VM, I can access the all the others VM content via their DNS when I'm outside my LAN (from my phone on 4G), but I can't reach them from my PC on the same lan. The connection times out (took too long to respond).

My port forwarding rules : see attachment 1
The firewall rules (automatically created) : see attachment 2
My current wan/lan setup : see attachment 3


The "443 rule" is my nginx reverse proxy (192.168.1.105). And everything is working when I'm outside my LAN (I also tried from an other ISP connection, and it's working). It simply doesn't work when I'm inside my LAN, and I can't find the option that will make it work (I can smell it ! It's one checkbox to tick ?!? ).


Any help would be appreciated, sorry again if this is an obvious thing. I couldn't find a way to properly formulate.


TL;DR : My dedicated nginx reverse proxy is working outside my LAN, doesn't work inside my LAN (connection times out).

 

[toomanylogins]

I am new to open sense as well with the same setup nginx reverse proxy and I can't get it working. I'm confused between the port forwarding and if you need to create rules on the WAN and on the interface that is connected to the reverse proxy. I can't seem to port 443 to forward to the nginx server let alone proxy the request. Any pointers to a firewall setup guide would be useful. Ive read docs but confused over whether you need rules ie wan to wan address then wan address the lan proxy address then lan proxy address  to lan webserver etc.

1. Does port forward 443 also need a inbound firewall rule as well ?
2. If you have 2 lan how to route between them ie reverse proxy on lan 1 to webserver on lan 2

Thanks
Paul

[Greelan]

1. Yes, but by default when you set up a port forward on OPNsense it creates a corresponding inbound firewall rule for you
2. Create a firewall rule on the LAN 1 interface allowing traffic from the nginx reverse proxy to LAN 2. If you already have an “allow to any” rule on LAN 1 then this additional rule won’t be necessary

If a proper setup still does not work, it may be that your ISP blocks inbound 443. You might have to ask them to remove the block

Re the OP’s post - sounds like a classic NAT reflection issue. This can be fixed under Firewall>Settings (and make sure your port forwards are not overriding that)

[toomanylogins]

Thanks for reply. I have managed to get the https port forward working from WAN nginx proxy on (orange) network. I had to create port forward NAT and an inbound rule from WAN address to nginx ip address.

However I am completely confused about forwarding from orange to port 80 on LAN. Nginx is listening on port 443 but then relays to port 80.

My confusion is caused by inbound and outbound. I assume that traffic is allowed out from orange so I do not need an outbound rule on orange interface.

I need an inbound rule on the LAN interface where source is the orange net or ip address to specific lan ip address ?
When I do this I dont get port option nginx is blocked.

I have a default allow any rule source LAN to any

I am not knew to firewalls having used windows firewall and smoothwall for many years but I am finding this  confusing.

Thanks for help

[Greelan]

Best to post screenshots of your rules

[toomanylogins]

I got it working by installing the nginx plugin and lets encrypt plugin. Did as follows.

1. Change default port on opnsense to 81 (system settings admin).
2. Install both plugins
3. follow these docs https://docs.opnsense.org/manual/how-tos/nginx.html Upstream server on port 80 NOT 443
4. Make sure you view sites on on port 80
5. Dont try 443 until you have a lets encrypt cert you go round in circles
6. Issue cert with LE plugin it needs port 80 to do this bit
7. Update the nginx with LE cert and it will proxy https 443 to http port 80

Cheers

[FingerlessGloves]

lets encrypt plugin, I recommend you use DNS instead of HTTP challenge is less prone to problems :-)

It supports a lot of DNS providers.

[toomanylogins]

Thanks, I have more than one dns but will look into it.
I spoke to soon. I have it working to view a site by domain name from my lan but looks like external users cannot get access. I am baffled by the firewall rules. Nginx ix not showing an incoming request in the access log but I have a firewall rule on wan address for 80 and 443 pointing to this.firewall. Which I think is correct ?
Paul

[touch1337]

Re the OP’s post - sounds like a classic NAT reflection issue. This can be fixed under Firewall>Settings (and make sure your port forwards are not overriding that)

I tried to enable all the options, it didn't change a thing.

1) Reflection for port forwards
2) Reflection for 1:1
3) Automatic outbound NAT for Reflection


The nat option from the port forward rule I made is set to "use system default", I tried to set it to "yes" or "no", it didn't change anything.

Is there a way to properly troubleshoot this situation ?

Edit : I also try to set the "split dns" option (OPNsense calls it "Overrides"). I simply put a wildcard entry "*" and add my domain, to point to my nginx reverse proxy IP (local IP). Didn't change a thing.

[touch1337]

I got it working in 5 minutes by installing PIhole and using their override DNS option.

It's insane how scuffed OPNsense feels...

Anyway, it's all good now, I hope the load balancing doesn't suck and works. Because that is the only option I need that my current router doesn't support.

Thank for the help.