OPNsense Forum

Archive => 18.7 Legacy Series => Topic started by: ccesario on December 14, 2018, 11:34:46 am

Title: Nginx Plugin :: Log Viewer
Post by: ccesario on December 14, 2018, 11:34:46 am
Hello @fabian,
I upgrade two appliances to 18.7.9 and I could see that nginx has log viewr (Nice, thanks for it!)

But when I try access any logs, my Browser crash (in both appliances). I supose that is bacuse the size log file.
looking the logs I could see the following.

Code: [Select]
[14-Dec-2018 00:01:31 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522266880 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:20 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522290512 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:24 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522308872 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:42 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522314600 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:43 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522314800 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:05:48 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522354280 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:15:23 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522830464 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 08:19:09 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1536664032 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42

DO you got similar problem ?

Regards

Carlos
Title: Re: Nginx Plugin :: Log Viewer
Post by: franco on December 14, 2018, 12:47:22 pm
Seems to read a huge log file all at once. PHP says no.


Cheers,
Franco
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 14, 2018, 01:28:55 pm
Yes Franco, it seems.
Maybe a limiter with auto-refresh can solve it (like a tail -20 logfile )

Regaards
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 14, 2018, 06:27:50 pm
This would not help,  If you have around 400M  (just separate: 402_653_184) on server side, it will definitely also kill your browser even if I pass that through in smaller pieces. For such big log files I would not recommend to use them in a web interface. In your case I would recommend the ELK stack, which is better optimized for such problems (pagination is a problem here because the amount of entries is not fixed and new pages may come every second).
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 14, 2018, 08:29:02 pm
Hi @fabian, yes, the log file is big, but Im running default nginx config, I did not touch in config.
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 14, 2018, 08:50:28 pm
nginx now has a log rotation which should prevent logs becoming that large. Maybe it will work tomorrow as it wasn't ever rotated yet ;)
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 17, 2018, 05:44:45 pm
Hi @fabian, you was rigth, the nginx rotate the logs.
But, even with log rotation, the interface crashes when try load the log file.

About the tail -f solution, I think that can be a good choice, even with a big file the browser/php does not crash.

Like this

https://github.com/pfsense/pfsense-packages/blob/903855b9b2d726f09594f6caccad8015b6063ca8/config/squid3/34/squid_monitor.php#L48-L66
https://github.com/pfsense/pfsense-packages/blob/903855b9b2d726f09594f6caccad8015b6063ca8/config/squid3/34/squid_monitor_data.php
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 17, 2018, 07:01:06 pm
for now, I have replaced the backend call by a line by line read, which may fix the problem.

https://github.com/opnsense/plugins/pull/1075

opnsense-patch -c plugins 537da9cbe58d5291c5963df8e72fda5e7f770eba
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 17, 2018, 07:43:39 pm
Patch applyed, but the browser still crash (tested in several browsers)  :(

# opnsense-patch -c plugins 537da9cbe58d5291c5963df8e72fda5e7f770eba
....
done
All patches have been applied successfully.  Have a nice day.


Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 17, 2018, 07:51:26 pm
ok but the backend works?
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 17, 2018, 07:57:20 pm
Yes, I deleted the error log and re-created it with  (currently has 10 lines) and I can see by GUI log viewer

Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 17, 2018, 08:04:05 pm
the patch only reduces some memory use of PHP but it sends the same data. Since I do not have such a big log, it will be probably very hard for me to reproduce.

Do you know if the crash happens while rendering (creating a long table) or already when reading the response?
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 17, 2018, 09:36:36 pm
Well, I don't know tell when the error happen. indeed I did not investigate it.
I will take a time for this..... and if you want,I can provide one of my log files to you reproduce the problem.

Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 17, 2018, 10:49:23 pm
A log file would be good, please take care that you do not violate any data protection laws - so at least replace the IPs by :: and 0.0.0.0.
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 18, 2018, 12:39:03 pm
Hi Fabian,
Could you please provide me an email to snd the file?
Best regards
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 18, 2018, 05:41:13 pm
sent a pm with a nextcloud link
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 18, 2018, 06:01:50 pm
File uploaded.
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 18, 2018, 07:06:17 pm
I looked into it - the API provides the data just fine, backbone + browser native functions take over the big JSON file without issues (only takes long - 3 seconds on my machine). So what is needed is a paging for not rendering 200k log lines at once into the browser window. What do you think is a reasonable default? 200 Lines?
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 19, 2018, 11:29:25 am
Hi @fabian,

Nice to know that API it os ok.
Well, I could suggest 100 lines by default with a dropdown to choose more or less linesand maybe a checkbox to disable or enable auto refresh. (It is only a suggestion)

Regards
Carlos
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 19, 2018, 07:25:03 pm
This commit will give you a new frontend (https://github.com/opnsense/plugins/pull/1080/commits (https://github.com/opnsense/plugins/pull/1080/commits)):

opnsense-patch -c plugins b8d15a8b50b2b723830eb2d2c0117668f67f3420
tested it with your file and it works on my machine.
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 20, 2018, 12:49:52 pm
Hi @Fabian,
Patch applyed, but the browser still crashed, but after some tries, the file is loaded, but no refresh/update data and when I try navigate  using pagination the browser crashe.

Regards
Carlos
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 20, 2018, 06:28:37 pm
then you probably do not have enough memory. I have tested it on a Linux machine with 8GB RAM on Firefox.
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 21, 2018, 12:55:30 am
No Fabian, I have tested with several machines.... And one of then hás 32gb mem.
Regards
Carlos
Title: Re: Nginx Plugin :: Log Viewer
Post by: fabian on December 21, 2018, 10:08:27 pm
This one is your file on my machine (2nd page if I counted it correctly)
Browser: Mozilla/5.0 (X11; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
Title: Re: Nginx Plugin :: Log Viewer
Post by: ccesario on December 22, 2018, 08:49:34 pm
Hi Fabian, I understand you.
I have tested it with "static" log file (the same that I sent you) . And its ok.
But the problem is when using the current file (in production) with many updates in the current log.

There are something that is broken it.
I do not know how is the implementation, but why not usage the same method that firewall/rules log viewer?

Regards
Carlos