OPNsense Forum

English Forums => Web Proxy Filtering and Caching => Topic started by: rene_ on September 25, 2020, 09:02:11 pm

Title: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: rene_ on September 25, 2020, 09:02:11 pm
Hello everyone,

as some of you requested this, I will write down, how I configured my Nginx, as a simple reverse Proxy (including HTTPS with letsencrypt, and Web Application Firewall enabled).


Step 1: Installation
You need to install the nginx and lets-encrypt plugins.

After that, configure you’re letsencrypt so that you get a valid SSL certificate for your service.
You need to use DNS-01 validation method, because nginx will use the port 80, and the lets-encrypt plugin is not able to use the modify the Nginx configuration for a successful validation.
When youre done, you can continue to Step 2.
(You can also use official paid certificates, if you have one, you need to import the CA, Cert and Key unter System → Trust)


Step 2: Configure Nginx

You need to be sure, that your OPNsense is not using port 80 or 443.
So you need to change the default port of your OPNsense webgui.
This can be done under “System → Settings → Administration”.
You also need to disable the HTTP Redirect.
Restart your firewall when done.

From now on, all steps are meant to configure under Services →Nginx → Configuration

2.1 Configure the upstream server

First of all, you need to configure your upstream server, this is the real server, where your web application runs on.
This could be any host on your LAN, DMZ or whatever.

To do so, navigate to Upstream → Upstream Server and click on the + in the right bottom corner.
Now enter a description, IP and port (80 – HTTP in most cases).
Use 1 as Server priority.

2.2 Configure the upstream

Next you need to configure the upstream, where you link your created upstream server.
You need to do that, because you could also configure multiple servers, for the same upstream for load balancing.
So what we configure, is a “load balancer” with just one host.

Therefore navigate to Upstream → Upstream and create one.
Chose a description, and link the upstream server you just created.
As load balancing algorithm, use weighted round robin.
Leave the rest as it is, if you don’t use HTTPS directly on your upstream server.


2.3 Configure the Location

As the next step, you need to configure the Location (URL) of your web application.
Navigate to HTTP(S) → Location and click on Add.
As URL Pattern, just use slash (/) and match type none.
URL rewriting should be nothing.
Define the Upstream server you created before and leave the rest as it is for now.

Later, you can configure the Web Application Firewall rule here.


2.4 Configure HTTP Server

The last step, to bring your web application online, is to configure the HTTP Server.
Navigate to HTTP(s) → HTTP Server and click on Add.
This should match your need in most cases:

HTTP Listen Port: 80
HTTPS Listen Port: 443
Server Name: The URL your applications listens to (for example: cloud.domain.com)
Locations: the location created in step 2.3
URL Rewriting: Nothing selected
TLS Certificate: The issued Lets-Encrypt or imported certificate for this host

Leave the rest as it is for now.

2.5 Apply changes
When your done, click on General Settings and then on Apply
Your nginx should now be ready to server your web application.
Be sure to have correct firewall rules (from wan to this device, port 80 & 443)


This tutorial is not finished yet, i will explain some steps more detailed and attach some screenshots the next days.
Hope this is helpful :-)
Title: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: rene_ on September 25, 2020, 09:02:42 pm
reserved for Web application firewall configuration
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: GreenMatter on September 26, 2020, 08:50:39 pm
Got exactly the same results. I will add here my WAF policy for Nextcloud. If you have something else, please post it.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 04, 2020, 05:41:03 am
How to let web server show real IP address at access.log?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: mimugmail on November 04, 2020, 07:33:06 am
Search in config for X-Forward
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 04, 2020, 09:58:50 am
Search in config for X-Forward
I choose X-Forward-for at Real IP and apply it.But it's still not show real IP.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on November 04, 2020, 05:15:43 pm
no. with this setting you direct nginx where to look for real client IP (in case its another proxy in front of nginx).
nginx already sends real client IP with X-Forwarded-For header.
you just need to enable this field on upstream log
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 05, 2020, 02:53:59 am
no. with this setting you direct nginx where to look for real client IP (in case its another proxy in front of nginx).
nginx already sends real client IP with X-Forwarded-For header.
you just need to enable this field on upstream log
So besides enabling X-Forward-For, should I set something on the web server? Or should I set other things on nginx?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on November 05, 2020, 05:19:25 am
no need to enable anything on opnsense\nginx
Code: [Select]
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
already in location template by default
just check log fields on upstream server, enable logging for X-Real-IP and/or X-Forwarded-For headers and check results
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 05, 2020, 06:07:13 am
I already enable X-Forward-For but web server access.log also show nginx server IP

Attachment is web server access.log
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on November 05, 2020, 07:07:08 am
have you disable "Real IP Source option" in GUI? (it changes client IP )
and..$proxy_add_x_forwarded_for is pretty tricky variable
what if you add X-Real-IP header to apache log and look at it for client IP? (again. dont use "Real IP Source" option in GUI)
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 05, 2020, 07:16:38 am
Please see attachment.
This Real IP option.What I need switch it?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on November 05, 2020, 07:19:17 am
"none"
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 05, 2020, 07:27:51 am
OK,I restart opnsense nginx but also show opnsense IP.Could I need setup any option?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on November 05, 2020, 07:43:14 am
have you try to add X-Real-IP to appache log?
is there any chance that its another proxy in front of opnsense\nginx?
what "Remote IP" shows nginx logs?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 05, 2020, 08:05:37 am
In the Opnsense/nginx log.The remote IP is show real IP.So I need change apache or nginx web server.Right?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on November 05, 2020, 08:30:07 am
if you set "Real IP Source" to none in GUI.
and enable logging of x-real-ip and x-forwarded-for headers on apache:
x-real-ip should show real client IP and
x-forwarded-for should show something like "clientIP,+opnsenseIP"
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: akong77 on November 05, 2020, 09:40:10 am
Thanks.I found log format problem.I fix it.It's all ok.Thank you very much.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: vikozo on November 05, 2020, 01:50:46 pm
Hello @Rene_

Thanks for this nice tutorial!
i did try to config the HAProxy  but it is not running
https://forum.opnsense.org/index.php?topic=19897.0

would nginx reverse Proxy be the alternative to use?

have a nice day
vinc
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: spetrillo on January 30, 2021, 07:17:03 am
Anyone got an example of the firewall rules needed for this?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: RamSense on January 30, 2021, 08:34:48 am
@spetrillo

For websites behind nginx go to: firewall - rule - wan - add a rule:
protocol IPv4 TCP/UDP - destination <this firewall> - port 443 (https) - description < something you like>
+
protocol IPv4 TCP/UDP - destination <this firewall> - port 80 (http) - description < something you like>

that should put the traffic to your opnsense nginx
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: spetrillo on January 30, 2021, 04:38:10 pm
@spetrillo

For websites behind nginx go to: firewall - rule - wan - add a rule:
protocol IPv4 TCP/UDP - destination <this firewall> - port 443 (https) - description < something you like>
+
protocol IPv4 TCP/UDP - destination <this firewall> - port 80 (http) - description < something you like>

that should put the traffic to your opnsense nginx

Is this good to go?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: RamSense on January 30, 2021, 05:12:13 pm
That looks correct.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: ddywz on February 12, 2021, 06:07:51 am
Thanks for the tutorial.  I have followed it and done everything correctly but the service won't start on nginx.  Something is not right on it.  When I try to enable nginx under general settings and try to start the service it won't start.  If I disable nginx on general settings the service starts (at least the green arrow) shows up but it cannot be stopped with the square icon.  I have to go on dashboard and then stop it from there. 

Not sure why it is doing this.  Do you need to reboot after you set this up?

Thanks,
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: toomanylogins on March 06, 2021, 06:32:38 pm
Can you explain the last line ?

Be sure to have correct firewall rules (from wan to this device, port 80 & 443)

What is this device ? I seem to get stuck here every time

Thanks
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: toomanylogins on March 07, 2021, 04:03:47 pm
I have also got mine working but I dont understand why the destination is any and not wan address on this rule.
I would have thought it should be wan to wan address, source any destination this firewall port https.  To my thinking the destination is the wan address which is the external ip ?
Please excuse if I have this all wrong.
Thanks
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on March 07, 2021, 04:51:01 pm
@toomanylogins
clarify the question please
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: toomanylogins on March 11, 2021, 04:23:47 pm
Hopefully this is a better explanation. I am confused by the log file. If I want to let email port 25 from my internal network 10.10.10.19 (orange) I have to create a rule as follows. See image of log file before the rule below where outbound email blocked


NIC Orange
Direction IN
Source 10.10.10.19 Port any ie my server
Destination Any ie outside world
Dest Port 25

To my way of thinking if I want to let email OUT of my system I'm confused why you have to create a rule where the direction is IN.  I cant be the only one confused by this.
Thanks
Paul
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on March 11, 2021, 04:52:17 pm
sorry, I still don't quite understand your configuration (use nginx to stream smtp traffic?)
in general, before a packet can leave the OPN, it must reach OPN. this requires an allowing incoming rule
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: dati on March 12, 2021, 10:46:04 am
Nice Tutorial!
But for general Testing in a Homelab it is a bit too complicated to set up. If you want to quickly add another Service for testing, you have to set up 4 different entrys in 4 submenus.
I think I have to take a look at HAProxy;)
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: michaelgo on June 03, 2021, 02:15:07 pm
thank you so much for this simple and straightforward guide
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: kimtaro on September 10, 2021, 06:57:51 pm
Hello,
I followed this tutorial to make NGINX as simple reverse proxy but it's doesn't work for me.
When I want to access to my server from outside it doesn't display anything.
I also try to connect my home assistant application but I had this issue :
"Server Error
Sorry, but something went wrong on our side.
There is nothing you can do except wanting until we fix the issue."
Can you help me please?
Thanks you.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: newbee on September 11, 2021, 05:16:41 am
reserved for Web application firewall configuration

bump

thanks for the guide
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: kimtaro on September 11, 2021, 05:31:49 pm
Someone can help me please??
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on September 11, 2021, 05:51:36 pm
@kimtaro
Quote
Sorry, but something went wrong on our side
server error error_page
can you please start a new topic and share your config and nginx error logs?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 25, 2021, 09:09:02 pm
Hi, thanks for the tutorial!

This is my first post on the forum, after reading through quite a few (archived) posts related to nginx and trying to match configuration snippets from conf files on the net with the options in the web GUI. I hope it is ok to post it here as a reply, instead of opening a new topic. Excuses for the length.

Is there a lot that I can do wrong when following the tutorial? I spent a couple of nights on it now, including removal and reinstallation of the nginx plugin and removing/redoing the configuration to the letter and with variations, with reboots in between for good measure. I can't seem to get it to work.

My network is quite simple (for my first post I also drew my first diagram at asciiflow.com, excuse my drawing capabilities) :

┌─────────────────┐
│                 │
│  GPON box       │
│                 │
└───┬─────────────┘
    │
    │
    │
    │
    │ IPv4 + IPv6
┌───┴─────────────┐
│                 │
│ OPNsense        │
│192.168.1.1      │
└───┬─────────────┘
    │
    │
    │
    │
┌───┴────────────┐
│switch          │
└───┬────────────┘
    │
    │
    │
    │
┌───┴────────────┐
│switch          │
└─────┬──────────┴────┐
      │               │
      │               │
      │               │
      │               │
┌─────┼───────────────┼───────────┐
│     │               │           │
│  ┌──┴─────┐     ┌───┴────┐      │
│  │YUNO1   │     │ YUNO2  │      │
│  │ .30    │     │  .31   │      │
│  └────────┘     └────────┘      │
│  PROXMOX                        │
└─────────────────────────────────┘




Without nginx as reverse proxy I can get one of the Yunohosts on the net via my ISP IPv4. My goal is to at least have the websites of multiple Yunohosts available at my public IP, hence nginx.

Steps so far:
I have installed the ACME plugin, but have not yet used it (my domainprovider is not supported by the DNS option)

I started out doing that for multiple servers and sites at once, now only for one.

Observations:

Some notes:

Thanks for bearing with me so far. I have no clue where to continue troubleshooting, and would appreciate any help!
 


Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: fabian on September 25, 2021, 10:00:52 pm
When your backend server is communicating via HTTPS, you have to enable it in the upstream. Otherwise, nginx will connect via HTTP to that port.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 11:42:48 am
When your backend server is communicating via HTTPS, you have to enable it in the upstream. Otherwise, nginx will connect via HTTP to that port.

Thanks for your quick reply!

TL;DR: I ticked 'Enable TLS', and experimented more after that did not verifiably solve the problem (not reachable, no log entries). I experimented more, also without result.

Long version:
I have been experimenting with ticking the 'Enable TLS (HTTPS)'-box under upstream, but that was without knowing whether it was needed or not. Now I won't turn it of, and am experimenting with and without 'TLS: Servername override' (giving it the identical name as the domain name for that actual upstream server vs empty).

In the 'Upstream Server'-config, I had one line for the upstream server, with its internal IP and 80 as port as per the  tutorial. I added a second 'upstream server' line, for port 443, and linked it in the upstream together with the upstream server for port 80 (after two separate upstreams did not work)

In the 'Location' config there's a tick box 'Force HTTPS'. Is the result of that setting that nginx will switch the connection from 80 to 443, instead of relying on the upstream server to do the redirect from 80 to 443? For the time being, I leave that box unchecked, as it is mentioned nowhere. The upstream server will switch to HTTPS / 443 anyway and most clients will connect via HTTPS as well (apart from Letsencrypt for the upstream server's ACME client).
I duplicated the 'Location' line as well, to be able to use both upstream configurations, so I got 2 'Location' lines, but then nginx would not restart, complaining about duplicate locations. That implies one of my upstreams is superfluous, so I reverted that and the second upstream as mentioned above.

In the 'HTTP Server' config I have one line, with one server name (the domain name that points to the external IPv4, and that I mention in the override in upstream when not leaving it empty).
If the actual upstream server got its own TLS certificate, is it necessary to make it available in OPNsense so I can pick it here under 'TLS certificate'?

When comparing this to any tutorial I am sure I making things needlessly complex, but things just don't get rolling with the simple setup (neither do they now...)

The crazy thing is, totally unrelated lines still turn up in the nginx access log, such as 302's on GETs to a deeplink on the upstream server's domain, or POSTs to a deeplink to non existing deeplink on the external IP (scanning for vulnarabilities, I guess, but I don't understand why they show up in the access log when there was no configuration for them in nginx).

In the mean time, the actual upstream server is accessible from the LAN using a custom hosts entry on my workstation,
- edit - I mean, at least the server is accessible in the LAN, so it is not that the upstream server itself has a problem.

Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 01:32:42 pm
There is progress! Anyway, there are some errors in the nginx error log that relate to the domain name of the upstream (gialinh.nl) :-)

When enabling TLS in the upstream, does that do something to the readability of the logging? I got log records from connections on the LAN-side with hostnames not known to me, which gives me an uneasy feeling:

Code: [Select]
*47 upstream SSL certificate verify error: (10:certificate has expired) while SSL handshaking to upstream, client: 192.168.1.122, server: gialinh.nl, request: "HEAD / HTTP/1.1", upstream: "https://192.168.1.30:443/", host: "tgizmkzpcubin"
*47 no live upstreams while connecting to upstream, client: 192.168.1.122, server: gialinh.nl, request: "HEAD / HTTP/1.1", upstream: "https://upstream670c050f850943b19359b3fa719f5a7b/", host: "tgizmkzpcubin"
*51 no live upstreams while connecting to upstream, client: 192.168.1.122, server: gialinh.nl, request: "HEAD / HTTP/1.1", upstream: "https://upstream670c050f850943b19359b3fa719f5a7b/", host: "kcqexiggecaupwx"
*53 upstream SSL certificate verify error: (10:certificate has expired) while SSL handshaking to upstream, client: 192.169.139.161, server: gialinh.nl, request: "GET /services/blog/2014/10/26/apk/ HTTP/1.0", upstream: "https://192.168.1.30:443/services/blog/2014/10/26/apk/", host: "osba.nl", referrer: "http://osba.nl/services/blog/2014/10/26/apk/"

Thinking about it, my TLS servername override in the upstream would be responsible for some of those symptoms, would it not? It overrides all requests for any of the (sub)domains at the actual upstream server, so that at some point there's a mismatch in the connection chain. I will be removing the TLS servername override from the upstream. I will change the 'HTTP(S) Server' configuration and add the other (sub)domains under 'Server Name'.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 01:33:44 pm
OK, seems to help, one way or another. I got my phon's IPv4 in the error log now. and in the access log as well.

Still no website on screen, owing to a bad gateway.

Curious thing: port 80 on the WAN side, via domain gialinh.nl, shows an error message (Server error, sorry, but something went wrong on our side. There is nothing you can do except waiting util we fix the issue | Web Application Protection by OPNsense).

I have two 'Upstream Server'-lines for 192.168.1.30, one for port 80 and one for port 443, both connected to this one upstream record. The upstream record is connected to the single 'HTTP(S) / Location' record, which in turn is connected to a single 'HTTP(S) / HTTP Server' record with in that record 8 (sub)domains.

Logging seems to suggest I'm getting close, this is in the error log:
Code: [Select]
*14 upstream SSL certificate verify error: (10:certificate has expired) while SSL handshaking to upstream, client: 178.227.96.232, server: gialinh.nl, request: "GET / HTTP/1.1", upstream: "https://192.168.1.30:443/", host: "gialinh.nl"
*14 SSL_do_handshake() failed (SSL: error:1408F10B:SSL routines:ssl3_get_record:wrong version number) while SSL handshaking to upstream, client: 178.227.96.232, server: gialinh.nl, request: "GET / HTTP/1.1", upstream: "https://192.168.1.30:80/", host: "gialinh.nl"
*14 no live upstreams while connecting to upstream, client: 178.227.96.232, server: gialinh.nl, request: "GET /favicon.ico HTTP/1.1", upstream: "https://upstream670c050f850943b19359b3fa719f5a7b/favicon.ico", host: "gialinh.nl", referrer: "http://gialinh.nl/"

There are some records in the access log as well, from my phone (178.227.96.232) and others, all get HTTP 502.

Any hint where to look?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 01:40:00 pm
Still getting closer.

On 'HTTP Server' I ticked 'HTTPS Only', because on closer look I noticed port 80 in some of the error logs for failed handshakes:
Code: [Select]
*5 SSL_do_handshake() failed (SSL: error:1408F10B:SSL routines:ssl3_get_record:wrong version number) while SSL handshaking to upstream, client: 178.227.96.232, server: gialinh.nl, request: "GET / HTTP/1.1", upstream: "https://192.168.1.30:80/", host: "gialinh.nl"
Besides that. in 'Upstream' I disabled TLSv1/TLSv1.1 because as far as I know, Yunohost without modification will only talk 1.2 or newer.

So far, the only requests that made it to the upstream server were lines like:
Code: [Select]
/var/log/nginx# tail -f access.log
192.168.1.1 - - [26/Sep/2021:11:53:50 +0000] "\x16\x03\x01\x00\xBD\x01\x00\x00\xB9\x03\x03\xD8\x95{\xE3\xD6XX\xFA9?\xFCz\xDA\xB9\x8A\x18\xE6G\x5C\x1E\x9A\xC2&\xF9k\xA3\xBF\xB0\xE5j%m\x00\x008\xC0,\xC00\x00\x9F\xCC\xA9\xCC\xA8\xCC\xAA\xC0+\xC0/\x00\x9E\xC0$\xC0(\x00k\xC0#\xC0'\x00g\xC0" 400 166 "-" "-"
(so, this line is _not_ logged by nginx on OPNsense, but on the upstream server).
Once forcing HTTPS, those disappeared. Good riddance, but now not any requests make it to 192.168.1.30 at all, for any of the domains.

In the access log there are no HTTP 502's anymore. Now all requests that make it thus far, are logged with HTTP 302.

On my phone, when trying to reach http://gialinh.nl, I do get redirected to https://gialinh.nl, but then I get a timeout. Since enabling 'HTTPS Only', the IP of my phone does not turn up in the logging anymore (not in error, nor in access, for sure not in upstream logging).

In HTTPS Server I changed 'Access Log Format' to 'Extended', hoping to scrape suggestions from the access log, but it does not seem to be of influence.

I am too far out of my depth to do a systematic troubleshooting and fall back to ticking boxes and reverting them when the situation gets worse. Thank you for reading my autobiography thus far, I am looking forward to any hints that can help me further.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: fabian on September 26, 2021, 06:24:28 pm
By the log I guess you try to access an http port with https.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 06:31:10 pm
Hi Fabian,

Thank you so much for your attention, the frustration is driving me crazy!

By the log I guess you try to access an http port with https.

I noticed the same, that is why I ticked 'HTTPS Only' which I could have written more clearly. It did have an effect, because those log records disappeared (and the "\x16\x03\x01\..." records at the upstream end), but I still do not get through to the upstream servers.

Presuming you do not mind spending more time on my problem: which information can help in troubleshooting, and which way of presenting is most helpful?

Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 06:32:15 pm
Just to be sure: I don't need NAT for port 80 and 443 to get this to work, do I?


I tried doing a packet capture on LAN for 192.168.1.30/24, but it started only once without capturing anything. Since that first time, I have not been able to make any capture on LAN for a specific host address: the screen flickers, and returns with the start button. When I don't specify a host address, or include WAN in the selection, packet capture and viewing works as expected.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on September 26, 2021, 08:55:40 pm
@wbk
hi
imho
it looks like http\https methods and ports messed up:
Quote
(\x00k\xC0#\xC0'\x00g\xC0" 400 166
its 400 response code to http connection attempt on https port

and it is possible that the connection to the upstream is not fully configured:
it is better to set up a separate upstream for each site and specify the name in the TLS: Servername override field (don't forget to enable TLS SNI forwarding in Location settings)
for the debugging, you can try to disable the TLS: Verify Certificate in upstream settings. you can enable it later
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 26, 2021, 10:11:42 pm
Hi Fright,

Thank you for your time and explanation!

it looks like http\https methods and ports messed up:
Quote
(\x00k\xC0#\xC0'\x00g\xC0" 400 166
its 400 response code to http connection attempt on https port
Ah! I looked up some of the strings, and found more people complaining about similarly looking log records (so I did not look at the details). It explains why they did not appear anymore after ticking the 'HTTPS Only' box: now all traffic from nginx is sent as HTTPS to the upstream server.

Quote
and it is possible that the connection to the upstream is not fully configured:
it is better to set up a separate upstream for each site and specify the name in the TLS: Servername override field (don't forget to enable TLS SNI forwarding in Location settings)
Do I understand correctly that I will get:

Quote
for the debugging, you can try to disable the TLS: Verify Certificate in upstream settings. you can enable it later
This setting is absent in my version of OPNsense. I do have 'TLS: Trusted Certificate' set to 'Nothing selected', is that the same setting?

If you can find the time to guide me in the above questions, I will be looking forward to your reply. In the mean time I try to change the configuration to reflect that situation.

Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 27, 2021, 06:29:15 am
Quote
and it is possible that the connection to the upstream is not fully configured:
it is better to set up a separate upstream for each site and specify the name in the TLS: Servername override field (don't forget to enable TLS SNI forwarding in Location settings)
Do I understand correctly that I will get:
  • 1 x HTTP(S) Location (duplicate locations / are not allowed) (I think I can only select 1 upstream here, which one?)

Last night I realized: I might need a unique path to distinguish a location for each website. Is that correct?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: fabian on September 27, 2021, 08:35:48 am
No, you can also have a different subdomain.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on September 27, 2021, 09:12:32 am
Quote
1 x HTTP(S) Location (duplicate locations / are not allowed)
sorry, why you think so?
you can create several Locations with the same URL Pattern pointing to different Upstreams with diiferent naxsi policies etc
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: fabian on September 27, 2021, 09:52:12 am
Yes, but you should not assign them to the same HTTP server. Otherwise the behavior is undefined or it will not even start.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on September 27, 2021, 10:41:33 am
yep. in general: separate Server for each subdomain + separate locations (for each) pointing to separate Upstreams with specific names in TLS: Servername override field
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on September 27, 2021, 03:13:36 pm
Hi, thanks both for your input!

Quote
1 x HTTP(S) Location (duplicate locations / are not allowed)
sorry, why you think so?
you can create several Locations with the same URL Pattern pointing to different Upstreams with diiferent naxsi policies etc
Yes, but you should not assign them to the same HTTP server. Otherwise the behavior is undefined or it will not even start.

I thought it was not possible to have multiple identical locations (with only '/' in my case), because it gave an error when I assigned more than one of them to a single HTTP Server record. That won't be necessary of course, when I make 8 HTTP Server-records, each with its own Server Name.

To prevent HTTP/HTTPS mismatches, should I tick "HTTPS Only" where available, and leave "HTTPS Listen port" empty in HTTP Server?
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on September 27, 2021, 03:34:16 pm
Quote
That won't be necessary of course, when I make 8 HTTP Server-records, each with its own Server Name
yes. in general (without knowing backend SNI config, certificates names and your upstream certs verifying requirements) it's more convenient to make separate Server for each subdomain

Quote
To prevent HTTP/HTTPS mismatches, should I tick "HTTPS Only" where available, and leave "HTTPS Listen port" empty in HTTP Server?
usually ports 80 and 443 are set on the Server and the "HTTPS Only"  option is enabled.
it is more important that the Location and Upstream and Upstream Server settings are correct (this is where you had an error message, if I understood correctly): the port specified in the Upstream Server settings must correspond to the protocol specified in the Upstream settings
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on October 03, 2021, 05:17:07 pm
Hi Fright, Fabian,

Thank you so much for the time you invested to explain this to me.

I still have not been able to make it work last week and postponed further experimentation to this weekend, but the weekend is almost over and I have not yet been able to return to the Nginx configuration.

For the time being I will be running with fewer websites and no reverse proxy.

I will return to the topic later on and in case I find the problem(s) and a working solution, I will post here for sure (as will I with more questions ;-) )

Thanks again!
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: @lex on December 25, 2021, 08:43:36 pm
Hello !

Any progress ?

I'm struggling on my side.

Thanks !
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: wbk on December 27, 2021, 06:13:58 pm
Any progress ?

I'm struggling on my side.

Sorry for not writing any more. I have not been able to match the labels in the web interface with tutorials for Nginx. I hate to admit I threw in the towel (always carry a towel) and found a low budget VPS to host the service that I intended to proxy :-(
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: ccj47 on December 18, 2022, 09:53:22 am
Hi, how do I set proxy_hide_header Upgrade;?

I tried to add this proxy_hide_header Upgrade; manually to /usr/local/etc/nginx/nginx.conf but of course it will be removed by the plugin.

background:
iOS Nextcloud app cannot access the Nextcloud hosted behind the nginx reverse proxy plugin of opnsense.
https://help.nextcloud.com/t/unable-to-login-access-to-nextcloud-server-21-from-both-ios-app-and-ios-web-browsers/111911/2

My current solution:
For the Apache server behind nginx, add Header unset Upgrade in the corresponding VirtualHost section.
Title: Re: [TUTORIAL] Nginx as simple reverse proxy with web application firewall and SSL
Post by: Fright on December 18, 2022, 01:37:06 pm
Hi
like https://forum.opnsense.org/index.php?topic=31234.0