OPNsense Forum
Archive => 18.7 Legacy Series => Topic started by: ccesario on December 14, 2018, 11:34:46 am
-
Hello @fabian,
I upgrade two appliances to 18.7.9 and I could see that nginx has log viewr (Nice, thanks for it!)
But when I try access any logs, my Browser crash (in both appliances). I supose that is bacuse the size log file.
looking the logs I could see the following.
[14-Dec-2018 00:01:31 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522266880 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:20 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522290512 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:24 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522308872 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:42 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522314600 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:43 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522314800 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:05:48 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522354280 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:15:23 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522830464 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 08:19:09 America/Sao_Paulo] PHP Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 1536664032 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
DO you got similar problem ?
Regards
Carlos
-
Seems to read a huge log file all at once. PHP says no.
Cheers,
Franco
-
Yes Franco, it seems.
Maybe a limiter with auto-refresh can solve it (like a tail -20 logfile )
Regaards
-
This would not help, If you have around 400M (just separate: 402_653_184) on server side, it will definitely also kill your browser even if I pass that through in smaller pieces. For such big log files I would not recommend to use them in a web interface. In your case I would recommend the ELK stack, which is better optimized for such problems (pagination is a problem here because the amount of entries is not fixed and new pages may come every second).
-
Hi @fabian, yes, the log file is big, but Im running default nginx config, I did not touch in config.
-
nginx now has a log rotation which should prevent logs becoming that large. Maybe it will work tomorrow as it wasn't ever rotated yet ;)
-
Hi @fabian, you was rigth, the nginx rotate the logs.
But, even with log rotation, the interface crashes when try load the log file.
About the tail -f solution, I think that can be a good choice, even with a big file the browser/php does not crash.
Like this
https://github.com/pfsense/pfsense-packages/blob/903855b9b2d726f09594f6caccad8015b6063ca8/config/squid3/34/squid_monitor.php#L48-L66
https://github.com/pfsense/pfsense-packages/blob/903855b9b2d726f09594f6caccad8015b6063ca8/config/squid3/34/squid_monitor_data.php
-
for now, I have replaced the backend call by a line by line read, which may fix the problem.
https://github.com/opnsense/plugins/pull/1075
opnsense-patch -c plugins 537da9cbe58d5291c5963df8e72fda5e7f770eba
-
Patch applyed, but the browser still crash (tested in several browsers) :(
# opnsense-patch -c plugins 537da9cbe58d5291c5963df8e72fda5e7f770eba
....
done
All patches have been applied successfully. Have a nice day.
-
ok but the backend works?
-
Yes, I deleted the error log and re-created it with (currently has 10 lines) and I can see by GUI log viewer
-
the patch only reduces some memory use of PHP but it sends the same data. Since I do not have such a big log, it will be probably very hard for me to reproduce.
Do you know if the crash happens while rendering (creating a long table) or already when reading the response?
-
Well, I don't know tell when the error happen. indeed I did not investigate it.
I will take a time for this..... and if you want,I can provide one of my log files to you reproduce the problem.
-
A log file would be good, please take care that you do not violate any data protection laws - so at least replace the IPs by :: and 0.0.0.0.
-
Hi Fabian,
Could you please provide me an email to snd the file?
Best regards
-
sent a pm with a nextcloud link
-
File uploaded.
-
I looked into it - the API provides the data just fine, backbone + browser native functions take over the big JSON file without issues (only takes long - 3 seconds on my machine). So what is needed is a paging for not rendering 200k log lines at once into the browser window. What do you think is a reasonable default? 200 Lines?
-
Hi @fabian,
Nice to know that API it os ok.
Well, I could suggest 100 lines by default with a dropdown to choose more or less linesand maybe a checkbox to disable or enable auto refresh. (It is only a suggestion)
Regards
Carlos
-
This commit will give you a new frontend (https://github.com/opnsense/plugins/pull/1080/commits (https://github.com/opnsense/plugins/pull/1080/commits)):
opnsense-patch -c plugins b8d15a8b50b2b723830eb2d2c0117668f67f3420
tested it with your file and it works on my machine.
-
Hi @Fabian,
Patch applyed, but the browser still crashed, but after some tries, the file is loaded, but no refresh/update data and when I try navigate using pagination the browser crashe.
Regards
Carlos
-
then you probably do not have enough memory. I have tested it on a Linux machine with 8GB RAM on Firefox.
-
No Fabian, I have tested with several machines.... And one of then hás 32gb mem.
Regards
Carlos
-
This one is your file on my machine (2nd page if I counted it correctly)
Browser: Mozilla/5.0 (X11; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
-
Hi Fabian, I understand you.
I have tested it with "static" log file (the same that I sent you) . And its ok.
But the problem is when using the current file (in production) with many updates in the current log.
There are something that is broken it.
I do not know how is the implementation, but why not usage the same method that firewall/rules log viewer?
Regards
Carlos