nginx reverse proxy: Real Client IP for internal request does not work

[sjjh]

Hello everybody,

I'm seeking some help with the correct configuration of the nginx as reverse proxy. It shall be possible on an upstream sever to receive the real client IP address, not only the IP address of the proxy. This shall work in both cases, for requests from the internet via WAN and for request from the LAN ("split DNS"). Using a public DNS server I created a DNS entry for the subdomain test.example.org pointing to the public WAN IP address of the OPNsense. Using the internal OPNSense Dnsmasq service I created a DNS entry for the subdomain test.example.org pointing to the IP address 10.68.0.1, which is an internal (VLAN) interface of the OPNSense.

To test the setup, I created a dummy upstream nginx sever (within the VLAN 10.63.10.0/24) with following log_format:
'$remote_addr $http_x_forwarded_for $http_x_real_ip - $remote_user [$time_local] '
Two exemplary log lines are:
10.63.10.254 51.195.103.74 51.195.103.74 - - [03/Mar/2022:20:11:57 +0100]
10.63.10.254 10.68.0.1 10.68.0.1 - - [03/Mar/2022:20:19:53 +0100]
The first line shows a request from the internet. The internal IP address of the OPNsense is logged as the remote_addr and the public IP of the client on the internet (51.195.103.74) is both logged as http_x_forwarded_for and http_x_real_ip. To me, that looks as I intended it. But the second log line shows a request from within the LAN. Here only the (internal) IP address of the OPNsense/reverse proxy is logged (10.68.0.1) instead of the real client IP.

Currently the Real IP Source setting in the nginx GUI is set to none, and I also tried various settings, with and without the trusted proxies setting -- without luck.
What am I doing wrong? Any hint welcome.

Thanks!
Simon

[fabian]

Real IP source and trusted proxy is if you are already behind a proxy and you need to extract the real IP address from the protocol.

Your case looks more like your OPNsense is internet facing and in that case, you don't have to mess with real IP source at all. The real IP address should be just forwarded to the backend by default (using the industry standard x-forwarded-for).

[sjjh]

Your case looks more like your OPNsense is internet facing and in that case, you don't have to mess with real IP source at all. The real IP address should be just forwarded to the backend by default (using the industry standard x-forwarded-for).

Yes, I would agree to that. And it indeed does work for requests coming from the internet to the OPNsense. But it does not work for requests coming from the internal LAN to the OPNsense. Threre I only see the IP address of the OPNsense itself, and not of the client.

Simon

[fabian]

NAT reflection?

[sjjh]

Sounds like an interesting concept, so far (as written above) I used the internal DNS server to redirect the URL to the internal IP address:

Using the internal OPNSense Dnsmasq service I created a DNS entry for the subdomain test.example.org pointing to the IP address 10.68.0.1, which is an internal (VLAN) interface of the OPNSense.

The IP address logged, is indeed the IP address given in the DNS entry.

Sorry, I'm not sure if I got your comment, is our setup wrong, should I prefer NAT reflection over (currently used) Split DNS? Or were you wondering if we use NAT reflection (AFAIK not)?

Simon

[fabian]

With NAT reflection you go outside and then back inside. This may cause, that you see an OPNsense IP address. If you have split DNS, everything is fine.

[sjjh]

Well, unfortunately neither nor. As far as I understand, we don't use NAT reflection, but Split DNS. Nevertheless is does not work.

On further analysis, it actually does work with a HTTP connection/server, but not with a HTTPS connection/server. I double checked the configuration of both "HTTP Server" settings in OPNSense nginx GUI, and they are identical: trusted proxy: 10.0.0./8 and Real IP Source: X-Forwarded-For.

In the log file on OPNSense nginx Remote IP:
10.68.0.1 (for HTTPS)
10.63.19.122 (for HTTP)
In the log file on the upstream nginx server with my extended log format ($remote_addr $http_x_forwarded_for $http_x_real_ip, as written in first post):
10.63.10.254 10.68.0.1 10.68.0.1 - - (for HTTPS)
10.63.10.254 10.63.19.122, 10.63.19.122 10.63.19.122 - -  (for HTTP)

I'm clueless. What did I do wrong, where is the mistake hidden, which setting shall I check?
thx Simon

[sjjh]

*push* any idea?

[cookiemonster]

I've been struggling with the same feature but in a different setup i.e. passing the client ip from haproxy to an upstream nginx server but on TCP. I've not solved it yet but I had to read on the proxy protocol https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/
Maybe something there for you if you haven't tried it.