OPNsense Forum
Archive => 20.7 Legacy Series => Topic started by: ruggerio on October 25, 2020, 01:21:34 pm
-
i got nginx at 80% as reverse proxy for dovecot running. 80% because i don't get the external ip of the client transferred to the mailserver.
if i enable proxy protocol, the connection will not be accepted. Does anybody have an idea? Eventually i was searching false, but google did not reply that much on imap on reverse proxy
all i would need is having the external ip in the logs of dovecot.
Thx,
Ruggerio
-
Can't see how this could be solved.
If you - for example - proxy http requests, the client IP will be the proxy (nginx) but with http there is the chance to set a Forwarded-For header. For IMAP I don't see that chance. It has to do with networking principles: the proxy is not only sending the requests to the IMAP server, it also needs to take care about the answers. He couldn't do this if the source IP would be the real client.
Maybe you can explain, why you want to reverse proxy IMAP with nginx? A simple port forward would be too easy? In this case you would see the real IP instantly.
-
i have this already running on haproxy without any problem. In dovecot, you have to enter there 2 special lines for haproxy. of course, this is primarly seen for loadbalancing reasons.
haproxy_trusted_networks = 192.168.0.0/24,192.168.1.0/24
haproxy_timeout = 10s
main goal was having http/s on nginx, as there is a free of charge waf included.
For the reason, not having haproxy and nginx for the same, i try to change to nginx.
Ruggerio
-
As far as I can see there is only a solution for HaProxy available.
The client IP needs to be sent as additional info.
Saw someone creating an extra logfile just for this information.
https://serverfault.com/questions/869863/nginx-mail-proxy-hides-clients-ip
-
nginx is an IMAP proxy (not available in the UI) but if you want to handle it via TCP, there is s proxy you can install on the IMAP server to handle the proxy protocol as well:
https://github.com/cloudflare/mmproxy
-
@fabian
dovecot is proxy-aware, so i think it makes sense to try to configure it without mmproxy
-
@ruggerio
what is your current opnsense\nginx conf for imap? stream?
for now when you turn "PROXY protocol" on in GUI for Stream Server it enables proxy protocol to upstream and proxy client. so nginx waits for proxy protocol in imap request.
try to edit nginx.conf and delete "proxy_protocol" directive after port number in "listen" directive. then restart nginx and check result.
for dovecont conf i think "haproxy = yes" directive is needed
-
Hi,
I tried with and without proxy protocol. in conf-files it's not inserted, if you don't click it in gui. without proxy-protocol, it works, i have the haproxy-config listening to 10143, which then dovecot returns:
Okt 26 10:28:08 stlucia dovecot[51]: imap-login: Error: haproxy: Client timed out (rip=192.168.3.1)
Okt 26 10:28:08 stlucia dovecot[51]: imap-login: Error: haproxy: Client timed out (rip=192.168.3.1)
Okt 26 10:28:08 stlucia dovecot[51]: imap-login: Error: haproxy: Client timed out (rip=192.168.3.1)
Okt 26 10:28:06 stlucia dovecot[51]: imap-login: Error: haproxy: Client timed out (rip=192.168.3.1)
Okt 26 10:28:06 stlucia dovecot[51]: imap-login: Error: haproxy: Client timed out (rip=192.168.3.1)
...still with the ip of the sense...
thx
ruggerio
-
no, i suggest to enable PROXY in GUI but delete some of config after that
i.e. when you enable PROXY in GUI stream server part will look like:
server {
listen 143 proxy_protocol;
listen [::]:143 proxy_protocol;
blah-blah-blah
proxy_protocol on;
}
you can try to make it like
server {
listen 143;
listen [::]:143;
blah-blah-blah
proxy_protocol on;
}
-
# servers
server {
listen 143 proxy_protocol;
listen [::]:143 proxy_protocol;
access_log /var/log/nginx/stream_d3e1b124-88e2-4744-9538-cceaf6f84ff1.access.log main;
error_log /var/log/nginx/stream_d3e1b124-88e2-4744-9538-cceaf6f84ff1.error.log info;
proxy_ssl off;
proxy_pass upstream3605708c54c0460ca656e8fbaeadabb9;
proxy_protocol on;
}
so, this should be correct?
-
proxy_protocol parameter in "listen" directive makes nginx to expect proxy protocol when client connects to 143 port. so try to delete this parameter.
only "proxy_protocol on" directive is needed to enable proxy when talking to upstream
-
thx, fright, i changed to this:
server {
listen 143;
listen [::]:143;
proxy_ssl off;
proxy_pass upstream3605708c54c0460ca656e8fbaeadabb9;
proxy_protocol on;
}
Now, at least it hast the same behaviour as sending directly to Port 143 with data streaming - means, that there is not yet a transformation about the external ip. x-forwarded-for cannot be used in that use-case, right? I read, that imap usually needs ID, which needs a manual implementation for a mail-section. But in fact, the connection now is accepted also on port 10143 for the upstream-server, which was not the case before.
-
not correct, what i wrote. i had an old server inside nginx, which did it wrong. changed it to right, now it works.
would be a cool change, having a checkbox in opnense or a selection under proxy protocol:
1) checkbox: use haproxy protocol
2) selection: instead of clicking the proxy protocol having the chose between standard or haproxy
In case of selecting haproxy, the enries in the services list (listen-part) shouldn't been placed, but still this one here: proxy_protocol on;
this would make perhaps much easier for mailservers. shall i file a CR on github? btw. this is tested with dovecot only. in my protocols i see now the external ips
Thx!
Ruggerio
-
would be a cool change, having a checkbox in opnense or a selection under proxy protocol:
IMHO it can be two PROXY-enable checkboxes: "accept proxy protocol" and "enable proxy proto to upstream".
you can FR for this but I got the impression that this is unlikely to work.
so that the changes do not disappear when you apply nginx config via gui you can insert stream server config hook in streams.conf template and stop using PROXY checkbox in GUI. add "proxy_protocol on;" via that hook
-
does somewhere exist kind of rtfm for this?
Thx,
Ruggerio
-
don't think so
you can look at https://forum.opnsense.org/index.php?topic=16595.msg87573#msg87573
as an example of adding a hook
in "stream server" case its streams.conf template in /usr/local/opnsense/service/templates/opnsense/nginx/
at the end of the file before last brace you can insert hook for server post-conf. like
{% endif%}
include {{ server['@uuid'] }}_post/*.conf;
}
{% endfor %}
{% endif %}
-
Thx! Gonna read this!
Roger
-
OK, gave it a try, changed the template and reloaded, added the hook according the rtfm, having proxy_protocol = on; with in, then the following problem occured: The entry is double.
I then removed the proxy_protocol-entries within the streams.conf on the listen-lines. still the same. There must be somewhere else an entry for this. Do you have an idea? proxy_protocol will be written into the streams from other places than streams.conf from template-directory?
Or, is there a parameter to give to nginx to accept the last loaded parameter only?
-
im a little confused.
did you turn off PROXY in GUI?
proxy_protocol parameter in listen directive is set from gui
{% if server.listen_port is defined %}
listen {{ server.listen_port }}{% if server.udp is defined and server.udp == '1' %} udp{% endif %}{% if tls_enabled %} ssl{% endif %}{% if server.proxy_protocol is defined and server.proxy_protocol == '1' %} proxy_protocol{% endif %};
listen [::]:{{ server.listen_port }}{% if server.udp is defined and server.udp == '1' %} udp{% endif %}{% if tls_enabled %} ssl{% endif %}{% if server.proxy_protocol is defined and server.proxy_protocol == '1' %} proxy_protocol{% endif %};
{% endif %}
if you add "proxy_protocol on;" directive via hook you need to disable PROXY in GUI
also
you need to remove "proxy_protocol" string from template or (better imho) make this change:
before:
proxy_protocol {% if server.proxy_protocol == '1' %}on{% else %}off{% endif %};after:
{% if server.proxy_protocol is defined and server.proxy_protocol == '1' %}
proxy_protocol on;
{% endif %}
-
Hi Fright,
Confused? Right, so i was...
Those were the lines i modified (listen-directive) and commented (proxy_protocoll) - but still i got the message. I also thought to have forgotten to uncheck proxy_protocol. Looking at the the code, it seems, the directive is written in each case, if its on in GUI, it will be set to on, if not, then it will be set to off.
It seems, that i have to prepare manually a nginx.conf, having those lines corrected or to set the proxy_protocol=on; each time, i modified it.
Thx.
-
so for the interested ones (if somebody could verify, pls) a working config (mucho bricollagio):
Prerequiste: proxy protocol is not enabled in WebGUI for streams!
1) create in streams.conf under /usr/local/opnsense/service/templates/OPNsense/Nginx the following entry:
include {{ server['@uuid'] }}_post/*.conf; right before the last } of the last %endfor - reload streams config in WebGUI
2) go to /usr/local/etc/enginx/nginx.conf and copy the uuid of the stream(s) which have been newly created. They begin as include as last line per stream, e.g. server {
listen 143;
listen [::]:143;
access_log /var/log/nginx/stream_1ef25291-7e82-4e66-a677-a4629270ff87.access.log main;
error_log /var/log/nginx/stream_1ef25291-7e82-4e66-a677-a4629270ff87.error.log info;
proxy_ssl off;
proxy_pass upstream3605708c54c0460ca656e8fbaeadabb9;
# proxy_protocol off;
include 1ef25291-7e82-4e66-a677-a4629270ff87_post/*.conf; ---> UUID here!
3) create manually a folder with the uuid copied from 2) under /usr/local/etc/enginx/
4) within the newly created directory from 3) create a file e.g. proxy_protcol.conf, only content ist proxy_protocol on;
5) within the file streams.conf (the same as from 1)) search this lineproxy_protocol {% if server.proxy_protocol == '1' %}on{% else %}off{% endif %}; and comment it.
6) copy this one here right below the commented line from 5) {% if server.proxy_protocol is defined and server.proxy_protocol == '1' %}
proxy_protocol on;
{% endif %}
apply configuration in WebGUI and reload service. You should be done.
Thx @Fright for you help!