OPNsense Forum
English Forums => Web Proxy Filtering and Caching => Topic started by: GreenMatter on September 13, 2020, 05:48:13 pm
-
As I've almost hijacked already solved thread (https://forum.opnsense.org/index.php?topic=16595.0) I'm copying over here the last and the most fruitful :-) answer:
Where to find location.conf, somewhere within "/usr/local/etc/nginx"?
/usr/local/opnsense/service/templates/OPNsense/Nginx/..
Still I'm not able to locate separate config file where I could enter missing headers and include in main config..
in the end of location.conf right before last brace insert new hook.
so before:
{% endif %}{# honeypot #}
}
after:
{% endif %}{# honeypot #}
include {{ location['@uuid'] }}_post/*.conf;
}
go the GUI and apply nginx config (not reload, apply. so opnsense apply new location.conf)
go to the
/usr/local/etc/nginx/nginx.conf and be sure that at the end of location blocks there is a line like:
include 248efebe-d2e2-401b-b93f-f4d9061bb18c_post/*.conf;
if so, you can create folder like "248efebe-d2e2-401b-b93f-f4d9061bb18c_post" in /usr/local/etc/enginx/,
place a file like "mylocationname.conf" with your directives and Apply nginx conf.
imo that shoud work, (I checked and replaced User-agent header via the hook. works:
set $new_user_agent "${http_user_agent} via nginx";
proxy_set_header User-Agent $new_user_agent;
)
at any step, something can go wrong. always make a backup
And now I have a few questions. Are all of these changes persistent across upgrades?
And in web UI, in location we can enable websocket (proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $proxy_connection;)
but if I need to have them in "overall" section as per recipe (https://www.onlyoffice.com/blog/2020/01/how-to-configure-reverse-proxy-for-nextcloud-and-onlyoffice-editors/):
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Forwarded-Host $http_host/editors;
server {
listen 80;
location / {
proxy_pass_header Server;
proxy_pass http://nextcloud/;
}
location /editors/ {
proxy_pass http://onlyoffice/;
}
}
Does it make practical difference whether they are in location section or above?
-
Are all of these changes persistent across upgrades?
nope. upgrade of nginx plugin will overwrite location.conf. so it will need to be restored (but i hope that @fabian will include location _post conf hook in next version. and may be it will use same foldername syntax ))
Does it make practical difference whether they are in location section or above?
no. location part will work. "overall" section is used just in order not to write it in all locations
imho location is the right place for proxy_set_header directive.
you can set proxy_set_header directive globally. but afaik global proxy_set_header directive will apply to location only if there is no other proxy_set_header directives in location block. so there is a chance that global proxy_set_header will not work in our case and you still have to add it to location
-
nope. upgrade of nginx plugin will overwrite location.conf. so it will need to be restored (but i hope that @fabian will include this hook in next version. and may be it will use same foldername syntax ))
If I'm right, there are already built-in hooks in "server's" section, so it might be the same for "location"?
But :-), wouldn't it have been better to have in web GUI, let's say in Advanced view (server and location), an option to add headers manually, by typing them in? As same as it is in OpenVPN server configuration... If I may suggest it.
-
so it might be the same for "location"?
"include {{ location['@uuid'] }}_post/*.conf;" is )
an option to add headers manually, by typing them in?
don't think so. ideally, there should be no "free" fields in GUI config
read "Safeguard user input"
https://docs.opnsense.org/development/guidelines/basics.html
so I highly doubt that such a request will not be declined
I have an idea about using hook and GUI, but I have a feeling that I have already tired the team with my requests and ideas)
-
don't think so. ideally, there should be no "free" fields in GUI config
read "Safeguard user input"
https://docs.opnsense.org/development/guidelines/basics.html (https://docs.opnsense.org/development/guidelines/basics.html)
so I highly doubt that such a request will not be declined
I have an idea about using hook and GUI, but I have a feeling that I have already tired the team with my requests and ideas)
Ok, understand it. Thus in perfect world we should have all in GUI named as headers are or alternatively, in their description 8)
-
why only headers?
nginx has a huge number of directives ;D
and if each of them try to include in GUI and to validate each one before adding it to the configuration, then the head will burst.
therefore hooks are added i think ;)
-
You must be right, I'm not a pro!
Anyway I have problem with URI to location matching. For example I want to open page being behind proxy https://mysubdomain.mydomain.com/onlyoffice
HTTPS server name is mysubdomain.mydomain.com
Location URL Pattern is /onlyoffice
URL Path Prefix / or empty
I can't access page, error 404.
But once I change location to root folder
HTTPS server name is onlyoffice.mydomain.com
Location URL Pattern is /
URL Path Prefix /
I'm able to access it through proxy, what am I doing wrong?
-
it looks more like a nginx question than opnsense one.
but tell more detail what the user is typing, what the request to the upstream should be (perhaps url rewrite is required).
in first and second cases you use different server names..
details of 404 errors can be viewed in the "HTTP Error Logs"
-
it looks more like a nginx question than opnsense one.
but tell more detail what the user is typing, what the request to the upstream should be (perhaps url rewrite is required).
in first and second cases you use different server names..
details of 404 errors can be viewed in the "HTTP Error Logs"
I gave up because https://stackoverflow.com/questions/36120467/nginx-not-matching-location-i-think
Problem there was same as mine. When location's name was: /onlyoffice/ and proxy_pass ended or URL Path Prefix with "/" it resulted with URI being added to server name (host) what caused error. Otherwise I ended up with other location /.
So I simply created another server and added location onlyoffice as "/" - root block.
BTW, is there any way/fancy redirections to achieve what I wanted initially? I mean one server with root location and other, "URI" locations being opened at their respective root folders...?
-
I still cannot understand your task exactly. which address to redirect to.
the example given is not entirely relevant. they are considering using the proxy_pass directive. the plugin uses upstreams and we will not be able to control the full address directly from the proxy_pass directive.
Need an exact task, what do you want to direct where.
If you only need to "cut" folder from uri and proxy request to root of another server you can use url rewrite (im using it for webdav client support to overcome cookie auth problems)
-
I'll try to describe it to my best :-)
Onlyoffice server is available via https://onlyoffice.mydomain.com (https://onlyoffice.mydomain.com)
Upstream server is configured 172.16.1.20:5443
Upstream to use TLS and LE cert.
Nextcloud server is available via http://next.mydomain.com (http://next.mydomain.com)
Upstream server is configured 172.16.1.2:80
Upstream to do not use TLS
As nextcloud doesn't have SSL certificate (and onlyoffice prefers it) I need to use reverse proxy to connect them.
Under one server name I cannot(?) have 2 root locations, so nextcloud was root location and onlyoffice in subfolder/URI.
And problem is/was that when proxying onlyoffice, its page was either not redirected at all (request went to root location) or redirected as 172.16.1.20:5443/onlyoffice (proxy hostname next.mydomain.com/onlyoffice) which doesn't exist. Request must goes to (location) root folder of onlyoffice server. I guessed I would need to use rewrite or redirection to achieve that but I couldn't find a solution..
-
that is, if I understand correctly, you want the user to go to one address (next.mydomain.com), and by going to /onlyoffice/.. to see the content from the site onlyoffice.mydomain.com. and whether it can be implemented using nginx.
Yes and no.
nginх will rewrite the url without a problem and send a request to another server (so request to next.mydomain.com/onlyoffice/foo goes to onlyoffice.mydomain.com/foo).
the question is what's next. if there is only a file in folder - no problem (for example, next.mydomain.com/onlyoffice/mydocfile.doc request will work). but if there is a page (like /onlyoffice/index.html) and it contains, for example, a picture (<img src = "/themes/OPNsense/assets/img/opnsense.png">) - thats all. end of the story. browser will attach a relative path to the server address and will request a picture without "/onlyoffice/" part. and request will go to the first server.
of course you can try to work around this with $http_referer (check referer and route request to onlyoffice if referer contains "onlyoffice"), but this requires testing and will not necessarily work smooth.
in short, if it is necessary for this to work stably, then it is necessary to make sure that the onlyoffice is actually in the /onlyoffice (ie onlyoffice.mydomain.com/onlyoffice ) then it shoud work smooth
maybe I missed something, but it seems like this
-
Thanks, ultimately I've decided to go for 2 servers with root locations... It works :-)
BTW, today when logging in to opnsense I was welcomed with completely full drive - usage of 108%!
and the reason was:
root@OPNsense:~ # cd / && du -ma | sort -nr | head -n 20
50463 .
32519 ./var
31071 ./var/log
30435 ./var/log/nginx
30430 ./var/log/nginx/ds1.xxxxxx.xx.error.log
Is it normal? Shall I simply disable Access Log Format in this particular HTTP server settings? Or can we set somewhere max. size of logs?
-
Is it normal?
of course not
whats in log?
obviously something is wrong with server and errors are poured into the log at a terrifying speed
-
whats in log?
As I connect to this OPNsense instance over VPN I was afraid of having this system completely crashed and lost access - half of services were already halted. Therefore I quickly deleted that file and restarted system. I could try to tail the log, but I wasn't sure of outcome of command on 30GB file...
Anyway, are there any size limits for error logs?
-
Anyway, are there any size limits for error logs?
https://github.com/opnsense/plugins/blob/master/www/nginx/src/opnsense/service/templates/OPNsense/Nginx/newsyslog.conf
not set.
you can edit template and add size limit (in kilobytes)
but I really think it's not about the newsyslog settings. you need to look at the log and figure out what is the reason for so many errors.
-
Anyway, are there any size limits for error logs?
you can edit template and add size limit (in kilobytes)
but I really think it's not about the newsyslog settings. you need to look at the log and figure out what is the reason for so many errors.
Does it look sensible and correct (10000k)?
root@OPNsense:/usr/local/etc/nginx # cat /usr/local/opnsense/service/templates/OPNsense/Nginx/newsyslog.conf
# logfilename [owner:group] mode count size when flags [/pid_file] [sig_num]
{% if helpers.exists('OPNsense.Nginx') %}
/var/log/nginx/*access.log www:www 640 14 10000 @T00 GZB /var/run/nginx.pid 30
/var/log/nginx/*error.log www:www 640 14 10000 @T00 GZB /var/run/nginx.pid 30
{% endif %}
And you are right, I should investigate it further, but like I said, my main task was not to loose connection :-)
Anyway, currently error log for that particular upstream/server stays empty; maybe it was just one off glitch...
-
Does it look sensible and correct (10000k)?
yep
my main task was not to loose connection :-)
agree )
-
I can't solve small issue: If I need an external access, I'm used to expose services running inside LAN on router's WAN interface but using random port numbers. Such port forwarding works fine but either upstream or Nginx keep rewriting random port to standard 443 and communication is lost. I mean when I have set forwarding i.e. port 4443 to 443: page https://server.domain.com:4443 gets rewritten after typing in logon credentials to https://server.domain.com/login and of course lost comms error is displayed. When I manually amend port to 4443 and and hit enter page is opened and user is logged in. Will "port_in_redirect off" in server block be permanent a solution? Any cons?
-
Will "port_in_redirect off" in server block be permanent a solution?
dont think so
gets rewritten after typing in logon credentials to https://server.domain.com/login
look at backend logs for responses after user authentication. i think that authentication form in /login page redirects user to absolute url after authentication.
if so you can try proxy_redirect directive in location block
(http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect)
(https://stackoverflow.com/questions/20254456/intercepting-backend-301-302-redirects-proxy-pass-and-rewriting-to-another-loc)
-
look at backend logs for responses after user authentication. i think that authentication form in /login page redirects user to absolute url after authentication. if so you can try proxy_redirect directive in location block (http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect (http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect)) (https://stackoverflow.com/questions/20254456/intercepting-backend-301-302-redirects-proxy-pass-and-rewriting-to-another-loc (https://stackoverflow.com/questions/20254456/intercepting-backend-301-302-redirects-proxy-pass-and-rewriting-to-another-loc))
In general, I'm confused here. Since port forwarding is done by firewall (fw 4443<->443 Nginx), thus for Nginx it's transparent. But looking at curl reply, in header there's:
location: https://server.domain.com/login (https://server.domain.com/login)
I've tried in URL rewriting to set
Source: $scheme://server.domain.com:4443/(.*)$
Destination: $scheme://$http_host/$1
Flag: Redirect
and activate it in location but that doesn't change a bit.... Does $http_host contain domain and port?
EDIT:
Since firewall/NAT forwarding is transparent for Nginx, Nginx is not aware of port, client is communicating on, so in my humble opinion there's no way to change header in location response, neither using redirect nor proxy_redirect.... Unfortunately
-
now i'm confused again. who is listening to which port. firewall? nginx? backend?
you have another firewall in front of opnsense?
if nginx listening on 443 then $scheme://server.domain.com:4443/(.*)$ makes no sense <- erroneous assumption due to misunderstanding of the network diagram
Does $http_host contain domain and port?
yes. should
no way to change header in location response
why? it should work imho. if after users auth backend server send 307 with location: https://server.domain.com/welcome.html, proxy_redirect will replace loaction with https://server.domain.com:4443/welcome.html for you. then client hit the 4443 port
-
why? it should work imho. if after users auth backend server send 307 with location: https://server.domain.com/welcome.html (https://server.domain.com/welcome.html), proxy_redirect will replace loaction with https://server.domain.com:4443/welcome.html (https://server.domain.com:4443/welcome.html) for you. then client hit the 4443 port
Ok, I'm not good at all in proxy stuff :-)
Nginx listens on 443, firewall opens WAN port 4443 and forwards traffic to Nginx port 443.
So, if it's possible, can I somehow configure such redirect in webgui or I must go for hook file? Could you show example for above case?
-
Nginx listens on 443, firewall opens WAN port 4443 and forwards traffic to Nginx port 443.
ok. it remains to understand which port the backend is listening to. and is there a url changing between user request and request from nginx to backend.
if the problem is only in the absence of a port in the redirect location then you can try
proxy_redirect https://server.domain.com/ https://server.domain.com:4443/;
I think it's better to start with explicit paths. if it works, you can pick up variables ($http_host or some)
can I somehow configure such redirect in webgui or I must go for hook file?
GUI does not contain this directive. use server or location hook
ps. I haven't checked it myself
-
ok. it remains to understand which port the backend is listening to. and is there a url changing between user request and request from nginx to backend.
if the problem is only in the absence of a port in the redirect location then you can try
proxy_redirect https://server.domain.com/ https://server.domain.com:4443/;
Backend listens on port 80.
Above proxy_redirect works in location block, but how to make "if" statement to kick in redirect only when client makes request on port 4443 which in turn is transparent for Nginx since firewall forwards such request to Nginx... So, as you mentioned it must be based on redirect code 307 or 302? Otherwise LAN clients can't login using regular https port...
Below is curl request/response, it's 302 redirect and it's the same regardless request origin (LAN/WAN):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 302
< server: nginx
< date: Wed, 23 Sep 2020 23:59:15 GMT
< content-type: text/html; charset=UTF-8
< content-length: 0
< cache-control: no-store, no-cache, must-revalidate
< content-security-policy: default-src 'self'; script-src 'self' style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
< location: https://server.domain.com/login
-
client makes request on port 4443 which in turn is transparent for Nginx since firewall forwards such request to Nginx
actually I got confused in the network diagram and said a stupid thing. nginx is aware of the initial request (if firewall only redirects the port and does not change the request). so, as I said, if an explicit directive works, you can pick up variables. in theory it should work like this
proxy_redirect https://server.domain.com/ https://$http_host/;
there are other options: make nginx listen on two SSL ports (443 and 4443) and make firewall redirect publicIP:4443<->nginxIP:4443 and insert $server_port variable.
or change the redirect location depending on the client's address (geo directive).
-
Thanks @Fright!
It works well with proxy_redirect and independently from WAN port number. And it is such a simple solution...
My goal is to set services in OPNsense mainly via webgui for ease of future maintenance. I used to have USG 4 Pro from Ubiquiti but that gateway's webgui functions were lacking seriously - most of a bit more advanced functions, had to be configured via json file (including NAT, OpenVPN and so on). 8)
One more thing, is it possible to use WAF whitelist? I mean If I set it to allow matching requests, all other requests will be rejected?
-
glad it works!
is it possible to use WAF whitelist? I mean If I set it to allow matching requests, all other requests will be rejected?
yes, if I understood you correctly. you can use whitelist policies to exclude certain rules from applied policies