OPNsense Forum

English Forums => Web Proxy Filtering and Caching => Topic started by: ensal on January 31, 2022, 10:35:00 PM

Title: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on January 31, 2022, 10:35:00 PM
I'm currently running 21.7.7 and am using the Nginx plugin to manage reverse proxy on my network. When I make a change in the web interface then check the nginx.conf file it is not writing my changes to there. This means the only way I've been able to do anything with it is to manually edit the nginx.conf file. So for things like increasing the max size of uploads or currently I'm attempting to add a new server to the reverse proxy but none of my changes are being written to it. Do I need to make a copy of the file for backup and delete it and hope it recreates the file and actually writes to it or is there something else that I should be checking here? I'm willing to provide whatever info is necessary.
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: fabian on February 01, 2022, 07:31:41 PM
Did you forget to regenerate the config?
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 02, 2022, 04:53:30 PM
In the past when I've made changes I'd just restart the plugin and that would work. Is that not the correct way to do it? I also just now tried selecting the refresh button to the right of the add button to see if that would have an effect and it did not. So I'm guessing I'm doing something wrong in regards to regenerating the config. How do I do it the correct way?
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 02, 2022, 05:08:23 PM
Hi
Nginx should be enabled (sorry, just in case  ;)) and you need to hit Apply in Services: Nginx: Configuration#General Settings
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 02, 2022, 08:08:06 PM
Yup I just tried that again and it's not working. Essentially what I'm currently trying to do is add another system for Reverse Proxy and it's not acknowledging that my rewrite rule is there so it just acts like the system doesn't exist. I also had to manually adjust max file size upload limits for my Nextcloud since it wouldn't change from the default 200m even with me setting it in the web interface.
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 02, 2022, 08:28:31 PM
maybe there are errors in the general log or the backend log when applying?
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 02, 2022, 09:38:56 PM
I've been searching for additional insight into this as well. I've checked in /var/log/nginx for any files that may give me details about that. I've also checked /var/log/system.log but found nothing there either. Any additional logs you recommend checking?
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 03, 2022, 03:05:19 PM
i would look in System: Log Files: Backend and Services: Nginx: Logs / Global Error
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 03, 2022, 05:34:45 PM
So here's what I found with that.

2022-02-03T11:27:42 configd.py[15718] [342270de-c733-4f42-a177-ac64122631dc] Script action stderr returned "b'[03-Feb-2022 11:27:42] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful\n\n[03-Feb-2022 11:27:42] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful\n\nnginx: the configuration file /usr/local/etc/nginx/nginx.co'"
2022-02-03T11:27:42 configd.py[15718] [9e7ae3bb-a8a3-4ef0-b493-baedabdbb34c] trigger config changed event
2022-02-03T11:27:42 configd.py[15718] [fac01a40-ae95-4085-9787-1da7fab52d53] request pf current overall table record count and table-entries limit
2022-02-03T11:27:42 configd.py[15718] [342270de-c733-4f42-a177-ac64122631dc] reloading nginx
2022-02-03T11:27:42 configd.py[15718] [64c0b534-a0d3-4242-a2d8-bf8bf1f07b7d] Inline action failed with OPNsense/Nginx OPNsense/Nginx/nginx.conf 'dict object' has no attribute 'serverentries' at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/template.py", line 267, in _generate content = j2_page.render(cnf_data) File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 1304, in render self.environment.handle_exception() File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 925, in handle_exception raise rewrite_traceback_stack(source=source) File "/usr/local/opnsense/service/modules/../templates/OPNsense/Nginx/nginx.conf", line 21, in top-level template code {% include "OPNsense/Nginx/http.conf" %} File "/usr/local/opnsense/service/modules/../templates/OPNsense/Nginx/http.conf", line 72, in top-level template code {% include "OPNsense/Nginx/upstream.conf" ignore missing with context %} File "/usr/local/opnsense/service/modules/../templates/OPNsense/Nginx/upstream.conf", line 15, in top-level template code {% for upstream_serveruuid in upstream.serverentries.split(',') %} File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 474, in getattr return getattr(obj, attribute) jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'serverentries' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 506, in execute return ph_inline_actions.execute(self, inline_act_parameters) File "/usr/local/opnsense/service/modules/ph_inline_actions.py", line 51, in execute filenames = tmpl.generate(parameters) File "/usr/local/opnsense/service/modules/template.py", line 344, in generate raise render_exception File "/usr/local/opnsense/service/modules/template.py", line 335, in generate for filen
2022-02-03T11:27:42 configd.py[15718] generate template container OPNsense/Nginx



So it's finding some issues I'm just not the most versed in establishing how to resolve the issue. I'm attempting to set this up as I've done all of my other Reverse Proxies in the past. I did have an issue after updating to 21.7.7 where my Nginx wouldn't launch since it was trying to bind to the wrong ports that I had not set it to and had to manually edit the file to make it work.
Any tips or ideas to go in would be great.
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 03, 2022, 06:08:45 PM
Quotejinja2.exceptions.UndefinedError: 'dict object' has no attribute 'serverentries'
it looks like the configuration for one or more of the upstreams is missing a reference to the upstream server(s)
Any chance that you tried manually editing config.xml?

QuoteI did have an issue after updating to 21.7.7 where my Nginx wouldn't launch
there was an error in the model migration function. now it is fixed. if there are backups of the config before the update, then we can try to fix config from backup with automatic "listen ports->listen address" migration.

if there are no backups left, then it remains only to try to iterate all upstreams and specify upstream servers in them
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 04, 2022, 03:04:23 AM
So I just realized the one system it shows in upstream server that does not exist in the web interface. Should I remove that from the nginx.conf file altogether or how should I go about getting it removed? I could also try removing the last of the remnant entries I have in the web interface that reference the old server that I have removed from the upstream settings.

As far as the error I got in the move to 21.7.7 I had to manually change the bind ports since it was trying to use 80 and 443 when I have Nginx set to bind to 8443. I do have backups of my entire opnsense based on backing it up to my Nextcloud dating back to what I believe is prior to the update and need for manual intervention so would that be useful? If not then I don't have system backups that are old enough to roll back to.
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 04, 2022, 05:58:11 AM
QuoteShould I remove that from the nginx.conf file altogether
no need to touch nginx.conf - it will be overwritten when the templates are applied successfully
QuoteI could also try removing the last of the remnant entries I have in the web interface that reference the old server that I have removed from the upstream settings.
yes. should work (or at least there will be another error  ;))

Quotehave backups of my entire opnsense based on backing it up to my Nextcloud dating back to what I believe is prior to the update and need for manual intervention so would that be useful?
yep, may work (better make new backup  before restoring old one. just in case)
so if you want to try with config restore:
try to apply migration fix:
# opnsense-patch -c plugins 6778f56
and if it is successfully applied, you can try with the backup restore.
but keep in mind that recovery does not allow you to select a specific plugin as a recovery area
you will have to select "OPNsense Additions" (you can uncheck the "reboot" flag), so that the settings of other plugins will also be restored. if this is unacceptable, then it is better to stick to plan a)
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 04, 2022, 09:43:10 PM
@Fright
I wanted to let you know that you are my hero!
Plan A worked immediately. Upon removing all disabled rules that referenced the old deleted server the nginx.conf file updated to include my new system that I've been wanting to add in as soon as I hit Apply on the main Nginx plugin page.
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 04, 2022, 11:03:21 PM
Now if I can just figure out why it's not  letting me adjust the max file size upload from 200m to 5000m I'd be all set. I've set the size adjustment under both of my inbound location entries for my Nextcloud. I've also set it on the http server entry as well but when hitting Apply it's setting it back to 200m in my nginx.conf file and I see it shows in the logs that the file is updated successfully
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 05, 2022, 06:16:34 AM
QuotePlan A worked immediately.
glad to know) thanks for the feedback
QuoteNow if I can just figure out why it's not  letting me adjust the max file size upload from 200m to 5000m
trying to set the values directly in nginx.conf?

works good for me if i set Maximum Body Size value in Server or Location GUI forms ("advanced mode" toggle should be "on" to see these fields)
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 05, 2022, 06:24:35 PM
Yeah for some reason setting it within the web interface it just won't set it to 5000m max body upload but editing it from the actual nginx.conf file does work. So not sure what the new issue is for that since here's the result from trying to set it from the web interface

2022-02-05T12:20:59 configd.py[15718] [72c5f666-a604-4786-a9b6-42d15ecbc8ca] Script action stderr returned "b'[05-Feb-2022 12:20:59] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful\n\n[05-Feb-2022 12:20:59] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful\n\nnginx: the configuration file /usr/local/etc/nginx/nginx.co'"
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 05, 2022, 08:11:24 PM
QuoteScript action stderr returned "b'[05-Feb-2022 12:20:59] NOTICE:
yep, such a line gets into the backend log when the configuration is successfully checked.

it seems that the behavior of the script has changed a little by nginx some time ago and now requires an additional flag (nginx_reload_quiet) so that the configuration check is silent
https://github.com/opnsense/ports/commit/5a05b2cf815d412adc73610ebceca0ca1cb762a1#diff-98eb80df33697b23b19c8b6753779cc7294e63ba9f94ed847661fcf095a94eb6R20

i will try and make pr if it helps

back to the question: I tried on 21.7.8 and the  'client_max_body_size 5000m;' directive is set successfully in the server and location section

wonder what you mentioned:
Quotewhen hitting Apply it's setting it back to 200m in my nginx.conf
but this is http block (and its no way to change it in gui). server and location blocks do not contain this directive unless they are set in the GUI. sorry, did you check server and location blocks after applying gui parameters?
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: ensal on February 12, 2022, 03:16:27 PM
Just to make sure I follow this. If I set it in the locations then I don't need to set it in the http server section? Sadly I don't have any larger files to test against it at the moment.
Next time I have a large file to upload to my Nextcloud that this is ultimately being set for I can post about it again.
Title: Re: Nginx Plugin Changes In Web Interface Aren't Working
Post by: Fright on February 12, 2022, 04:42:01 PM
QuoteJust to make sure I follow this.
if you want to apply this max size to whole server then you can cet it at server level. if you want to keep default limit (200M) for server and increase it for just one dir then you can set it at location level.
that is:
Quoteset it in the locations then I don't need to set it in the http server section?
yes  ;)
QuoteI don't have any larger files to test against
https://pinetools.com/random-file-generator  :D