NGINX Issue with multiple Virtual IPs

[gdur]

I'm trying to setup nginx reverse proxy having multiple virtual IP's and I wonder what is the appropriate way to catch traffic entering a specific virtual IP. Nginx is running and both port 443 and 80 are open, checked that in shell with sockstat. After starting nginx no traffic was catched so looking at life log I saw that traffic was blocked by the "Default deny / state violation rule". In order to direct HTTP(S) traffic to nginx I made a NAT port forward rule to the local LAN interface. That kind of works but it don't believe this is the right way. Any suggestions?

[cookiemonster]

Did you remember to change the port of OPN UI so that it doesn't use those ports?

[gdur]

Yes I sure did. The OPN UI service is using different ports...

[cookiemonster]

OK. Best to describe the setup then, to see what ip are the virtual and real servers and ports. It's a bit unclear right now. The description is quite generic. Then descrive the problem or what you want to achieve. For instance what do you mean by catch traffic? Is it logging, something else?

[gdur]

I try to clarify what I would like to achieve. I have multiple virtual, public IP's on my OPNSense box. Currently I host various websites on various servers behind the firewall. HTTP(S) traffic is routed to these webservers using NAT redirect rules and that's working fine. I now want to catch web traffic by NGINX on OPNSense and than forward traffic to the appropriate server. So I changed the 443 port, which was used before by the UI, to 7443 but HTTP(S) traffic from the outside pointed at one of the virtual IP's is not being catched by NGINX. What should be the way to make NGINX catching this HTTP(S) traffic?

[cookiemonster]

I don't know what virtual ips are in this context. OPN allows to create virtual ips but they aren't public ones, so I fail to understand what you mean. Also I don't know what you mean by catching traffic. I guess you mean route the traffic..
Having said that, setting up nginx in OPN is a case of defining the real servers, the upstreams, etc. Most simple scenarios can be done by the UI. See this https://forum.opnsense.org/index.php?topic=24778.0 for starters.

[gdur]

I have an IP block of 16 public IP addresses and assigned these to WAN via Interfaces->Virtual IPs->Settings in order to make them reachable for the outside world. Hence the modified subject line.Each of these IPs belong to a different web address. Under current situation once an URL has been requested belonging to one of these addresses it will be forwarded to the associated server behind the firewall. Now I actually want NGINX to listen to incoming HTTP(S) traffic belonging to a specific public (Virtual) IP address/URL's and let NGINX take care of the forwarding while making use of the extra offered protection. As Virtual IPs are part of WAN and allowing standard HTTP(S) on WAN now results in an error:

Server Error
Sorry, but something went wrong on our side.
There is nothing you can do except waiting until we fix the issue.
Web Application Protection by OPNsense

This is obviously an unwanted situation and I'm stuck at this point.
Not sure whether this clarifies and I hope you get what I'm doing wrong.

[cookiemonster]

Clear now.
Although I have two public ips they are two physical lines I only have one using OPN so I haven't tried your setup. The part I don't know about is the routing setup in OPN for virtual ips, so my suggestion my not work.
Can you share your nginx config for one real server and your firewall rules that you have setup?

[gdur]

Hi Cookiemonster,
Today I figured out that it is definitely not a NGINX issue but a Virtual IP setting. So I have to say to myself RTFM.
It's just too long ago that I've set-up my OPNSense box and now I forgot to read the documentation on Virtual IPs again. I had my v-ips configured in the default/ IP-Alias mode. In this case I never had a problem because I redirected all incoming traffic anyway, this is what the documentation says:

Either Network or Single address, only has affect when creating NAT rules, where Proxy ARP and Other combined with Expansion will generate separate addresses for all items in the netmask.

After reading this statement a couple of times I understood that most likely a virtual IP is not listening to any port request. A port scan check confirmed, port 443 on the target virtual interface was closed.
Another problem though occurred right away:

Virtual IP mode may not be changed for an existing entry. but I will make another topic for that.

but I will make a new topic for this as it is unclear how to deal with this on a production machine.

[cookiemonster]

Glad you're closer. It's an interesting setup, I'd like to know how you end up solving it in OPN. I'll read the other threads