Web Server Instructions / Let's Encrypt / Nginx

[Deku2]

I'm needing some guidance on setting up a web server behind OPNSense.  Initially I just did a port forward, but I want TLS.  So I installed the Let's Encrypt Plugin on OPNSense, but I'm not sure how this works with port forwarding as the server itself needs the cert as it does the encryption exchange.  I don't want to open the web server to the world (I have a Alias defined IP ACL).  So just installing Certbot on the webserver is not an option unless it somehow interacts with OPNSense to allow the temporary proxy.  I also don't want to set up some method that copies the cert from OPNSense to the webserver as that would involve too many potential problems and security issues.

I'm also interested in putting a WAF in front of the web server, though this is not required.  So maybe some nginx method is possible, where the Let's Encrypt on OPNSense is served to the Nginx plugin which acts as a front end to my webserver?  Then the Let's Encrypt plugin has something called a HAProxy, so maybe that's the solution?  I'm finding documentation on these aspects of configuration very limited for my situation and I could really use the help trying to get this set up correctly.  Thank you for any help you can provide. 

As an additional note, I need to be able to access it both internally and externally.  DNS will resolve to the external IP, but I don't know if I need to do some reflection or anything since it would need to go out and then back in.

[fabian]

First of all, HAProxy and nginx are both load balancers and reverse proxies available on OPNsense. Both can terminate your TLS connections and handle the ACME challenge using the acme client plugin. The difference comes when it's about your goal. The nginx plugin has more features and can do more (for example it has a WAF included) while the HAProxy is more optimized for fastest load balancing.

[fabian]

If you use nginx or haproxy, you don't need NAT reflection btw.

[Deku2]

Thanks @fabian.  I think I got the Let's Encrypt working fine.  The nginx is confusing as heck and I don't quite understand what I'm suppose to do.  I've created an upstream (whatever that is) and upstream server (whatever that is), a location (whatever that's for), an http server (this actually makes sense I think - the box I'm sending the traffic to).  I haven't even gotten into the ACL and WAF.  Sure wish it was as easy as a port redirect with a few options.  I'm trying to read through the site's instructions, but they seem to be missing some pieces and don't really explain what is being configured and how it's all supposed to work together.

You have https://wiki.opnsense.org/manual/reverse_proxy.html and I'm like, yes! I want to do #1 or just start with#2 and work toward #1, but it doesn't actually tell you how to do it.  So I've been trying to follow this load balancing doc, even though I'm not load balancing, but it's just not working.  https://wiki.opnsense.org/manual/how-tos/nginx.html

[fabian]

Upstream + Upstream Server: Where to forward the request to
Location + HTTP server: Configuration what to do with the request (a location binds a URL to a configuration how to handle it)
HTTP server (additional) listen sockets

[Deku2]

Still not getting this to work.  I can see the firewall is letting it through on the external port, but I see no activity in the http logs in nginx.  Here are my settings using just strait http (externally port 8888 for testing) to avoid any complication until I get it figured out.

[fabian]

the location url pattern you probably meant is just "/" (without the qoutes and no match type. Why you don't get any log lines I have no idea.

[Deku2]

The location pattern got me some logging.   Almost there I think. Working on a log error.

[fabian]

this sounds good because it means that nginx is trying to connect to your server but fails. You should try to use curl from the command line to check if it is reachable.

Maybe you have a firewall issue (the server blocks the incoming connection) or incorrectly configured listen parameter on the web server, so nginx cannot reach it. You may also try to do a packet capture on the interface where the server is connected to see why the TCP connection fails (I guess no ACK segments are coming back which leads to retransmissions and then the timeout).

[Deku2]

I got http working using port 8888 on the "http server", but not 80.  Tried doing https on 4444, but it's not working either.  It doesn't seem to be restarting nginx properly as under Firewall Sockets, it still shows it bound to port 8888, even though I've changed it and added https.  Tried Stop / Start via the dashboard too.

[Deku2]

@fabian I figured out the issue and I'd suggest some changes to the plugin.

When you're on one of the settings, such as HTTP Server, and click "Save Changes" it doesn't actually save anything.  When you restart nginx, you still have the same settings.  This can be verified by looking at the nginx.cfg.  You have to go the General Settings and click Apply before they'll actually be saved.  This is why I was ripping my hair out as it is assumed that Save Changes and a service restart should apply the new settings - it does not.  Not until you click Apply on the General Settings.  If it requires an Apply afterward, it should do the banner like on the firewall or something or prompt the user if they want to apply the settings.

[Deku2]

Seems you can't delete an IP ACL once it's created.  It would also be nice if the IP ACL could utilize the Firewall Alias groups.

[Deku2]

fabian, how do you turn off HSTS Strict-Transport-Security for nginx?  I've tried several things, but can't seem to find the option to disable this.

[fabian]

I got http working using port 8888 on the "http server", but not 80.  Tried doing https on 4444, but it's not working either.  It doesn't seem to be restarting nginx properly as under Firewall Sockets, it still shows it bound to port 8888, even though I've changed it and added https.  Tried Stop / Start via the dashboard too.

When you're on one of the settings, such as HTTP Server, and click "Save Changes" it doesn't actually save anything.  When you restart nginx, you still have the same settings.  This can be verified by looking at the nginx.cfg.  You have to go the General Settings and click Apply before they'll actually be saved.  This is why I was ripping my hair out as it is assumed that Save Changes and a service restart should apply the new settings - it does not.

Have you tried the reload button beside the add button - this one reloads the config with a reconfiguration (dashbord or top button will NOT reconfigure).
Also port 80 may not work because it can be in use by the web interface.

If it requires an Apply afterward, it should do the banner like on the firewall or something or prompt the user if they want to apply the settings.

I don't want it. I have the reconfigure button and apply does apply directly.

Seems you can't delete an IP ACL once it's created.  It would also be nice if the IP ACL could utilize the Firewall Alias groups.

If it cannot be deleted, it is probably a bug. Aliases can be supported but they are really hard to implement here since the model does not support that (even the ip acl is a custom component).

fabian, how do you turn off HSTS Strict-Transport-Security for nginx?  I've tried several things, but can't seem to find the option to disable this.

It is enabled when you enable HTTPS as well as you can configure it under security headers.

[Deku2]

It is not apparent that the little refresh icon is suppose to save the configuration.  If the button that says "Save changes" doesn't save the changes, why would a user click on a refresh icon that has no tooltip.  It would be really helpful if you made the Save changes button actually update the configuration.

Personally, I think HSTS should be off by default.  It's too easy to mess up when testing and you're unable to get yourself corrected without digging into your browser.  But if you're going to have it enabled, I think it would be very helpful to make it easy to disable.  It's clearly labeled in the System:Settings:Administration "Enable HTTP Strict Transport Security" but I can't find it in nginx.  There is a place in the Location that says "Force HTTPS" but disabling it doesn't appear to change HSTS, nor does the option in HTTP Server that says "HTTPS Only".  It makes sense that it might be in the Security Headers section, but I don't see where it's listed.  In fact, turning off the Security Policy has no effect on HSTS.  There is the option for "Strict Transport Security: Time" and "Strict Transport Security: Include Subdomains", but nothing to actually turn it off.  I guess I could set Time to 0?  I could manually edit the nginx.cfg, but I want this to be able to survive config changes via the gui and opnsense updates.