[SOLVED] Configure lighttpd to log real client IP behind reverse proxy?

[Greelan]

I run the OPNsense webserver behind a nginx reverse proxy. I have configured the X-Forwarded-For header in nginx to pass the real IP of the client connecting to the reverse proxy, but OPNsense does not register that.

To get OPNsense to do so, I understand from research that I need to enable the "mod_extforward" module in lighttpd and specify the IP(s) of the reverse proxy. Something like this:

Code Select Expand

server.modules += ( "mod_extforward" )
extforward.forwarder = (
     "xxx.xxx.xxx.xxx" => "trust",
     "xxx.xxx.xxx.xxx" => "trust"
)

My question is where is best to do this in OPNsense, so that it will survive updates. I suspect it is best not to do it in /usr/local/etc/lighttpd/lighttpd.conf? Should I:

• add it to /usr/local/etc/lighttpd/modules.conf?
• create a separate conf file in /usr/local/etc/lighttpd/conf.d/? In that case what is the best way to load it, eg adding an include directive to modules.conf?

Thanks

[franco]

Hi Greelan,

We never had to facilitate plugin behaviour for lighttpd configuration before, but adding a "drop in" directory would make sense unless a simple GUI option could be added if useful for others (checkbox-based). If we need to add data to a key somewhere that would make more sense without a GUI first (and a plugin could be created to add GUI parts for it if needed).

To test your changes you need to edit /usr/local/etc/inc/plugins.inc.d/webgui.inc for now. If it works I can look into include directory incusion.


Cheers,
Franco

[Greelan]

Thanks Franco, I will look into that.

[Greelan]

Yup, that worked.

Guided by the lighttpd docs (https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModExtForward), I included the mod_extforward module in the server.modules list, and added extforward.headers and extforward.forwarder directives.

Extract from webgui.inc from line 236:

Code Select Expand

## modules to load                                                                     
server.modules              =   (                                                       
  "mod_access", "mod_expire", "mod_compress", "mod_redirect", "mod_setenv",             
  "mod_cgi", "mod_fastcgi","mod_alias", "mod_rewrite", "mod_openssl" {$lighty_modules}, "mod_extforward"
)                                                                                       
                                                                                       
extforward.headers = ( "X-Forwarded-For", "Forwarded-For", "X-Real-IP" )
                                                                                       
extforward.forwarder = (                                                               
     "172.16.66.5" => "trust",                                                         
     "fdfd:2553:8868:66:216:3eff:feeb:8e62" => "trust"                                 
)                                                                                       

Note that I added the mod_extfoward module at the end of the server.modules list so that, if mod_accesslog is enabled through the $lighty_modules variable (reflecting that access logging is turned on in the web GUI), the mod_extforward module will be enabled afterwards, as stipulated in the lighttpd docs.

I also had to add the extforward.headers directive, because I found that the default (which is to search the "X-Forwarded-For" and "Forwarded-For" headers when the directive is empty) wasn't returning the real client IP despite X-Forwarded-For being set in my nginx config to $proxy_add_x_forwarded_for. So I added X-Real-IP, which is also set in my nginx config, and that worked. I will have to figure out separately the X-Forwarded-For issue with nginx.

Regarding the extforward.forwarder directive, that would obviously be the data item for the GUI, ie the user would be asked to enter the IP(s) of the reverse proxy. I guess the GUI could have both that, and an option to select to load the mod_extforward module (like for the mod_accesslog module), or alternatively the mod_extforward module could just be enabled by default and the only item in the GUI would be the ability to enter the IPs if the user is using a reverse proxy. I assume that the mod_extforward module, even if enabled, would just do nothing if there are no trusted IPs set in extforward.forwarder.

[Greelan]

Update: I discovered by changing the lighttpd logging format that it is indeed receiving the correct X-Forwarded-For header from my nginx reverse proxy.

The issue is that the default behaviour for extforward.headers, as explained in the lighttpd docs, has a bug: https://redmine.lighttpd.net/issues/3051#change-12200.

This was fixed a few days ago in the upstream. In the meantime a solution is, as I did, to manually specify the extforward.headers to include X-Forwarded-For (and anything else relevant). I've tested this by only specifying X-Forwarded-For and not X-Real-IP, and it works correctly.

[Greelan]

@franco, should I create a request on GitHub for this?

[Greelan]

Hi @franco, anything I can do to further progress this? I realise it is likely not a high priority, but I do like seeing the real IP in my logs, even if at the moment this requires reconfiguring the webgui plugin after each update [emoji3]

[gstrauss]

Just a follow-up that this was fixed in lighttpd-1.4.59 and is in OPNsense 21.1.1 (released Feb 10)
https://opnsense.org/opnsense-21-1-1-released/

[Greelan]

The bug in lighttpd's default behaviour for the mod_extforward module has been fixed and that fix is in OPNsense, but the implementation of that module in OPNsense has not yet occurred: https://github.com/opnsense/core/issues/4638

[Greelan]

PR created with simple implementation of solution: https://github.com/opnsense/core/pull/4833

[Greelan]

More generic PR created: https://github.com/opnsense/core/pull/4837