OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • Archive »
  • 18.7 Legacy Series »
  • Nginx Plugin :: Log Viewer
« previous next »
  • Print
Pages: [1] 2

Author Topic: Nginx Plugin :: Log Viewer  (Read 2844 times)

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Nginx Plugin :: Log Viewer
« on: December 14, 2018, 11:34:46 am »
Hello @fabian,
I upgrade two appliances to 18.7.9 and I could see that nginx has log viewr (Nice, thanks for it!)

But when I try access any logs, my Browser crash (in both appliances). I supose that is bacuse the size log file.
looking the logs I could see the following.

Code: [Select]
[14-Dec-2018 00:01:31 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522266880 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:20 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522290512 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:24 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522308872 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:42 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522314600 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:02:43 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522314800 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:05:48 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522354280 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 00:15:23 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1522830464 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42
[14-Dec-2018 08:19:09 America/Sao_Paulo] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 1536664032 bytes) in /usr/local/opnsense/mvc/app/library/OPNsense/Nginx/AccessLogParser.php on line 42

DO you got similar problem ?

Regards

Carlos
Logged

franco

  • Administrator
  • Hero Member
  • *****
  • Posts: 10056
  • Karma: 762
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #1 on: December 14, 2018, 12:47:22 pm »
Seems to read a huge log file all at once. PHP says no.


Cheers,
Franco
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #2 on: December 14, 2018, 01:28:55 pm »
Yes Franco, it seems.
Maybe a limiter with auto-refresh can solve it (like a tail -20 logfile )

Regaards
Logged

fabian

  • Hero Member
  • *****
  • Posts: 2436
  • Karma: 171
  • OPNsense Contributor (Language, VPN, Proxy, etc.)
    • View Profile
    • Personal Homepage
Re: Nginx Plugin :: Log Viewer
« Reply #3 on: December 14, 2018, 06:27:50 pm »
This would not help,  If you have around 400M  (just separate: 402_653_184) on server side, it will definitely also kill your browser even if I pass that through in smaller pieces. For such big log files I would not recommend to use them in a web interface. In your case I would recommend the ELK stack, which is better optimized for such problems (pagination is a problem here because the amount of entries is not fixed and new pages may come every second).
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #4 on: December 14, 2018, 08:29:02 pm »
Hi @fabian, yes, the log file is big, but Im running default nginx config, I did not touch in config.
Logged

fabian

  • Hero Member
  • *****
  • Posts: 2436
  • Karma: 171
  • OPNsense Contributor (Language, VPN, Proxy, etc.)
    • View Profile
    • Personal Homepage
Re: Nginx Plugin :: Log Viewer
« Reply #5 on: December 14, 2018, 08:50:28 pm »
nginx now has a log rotation which should prevent logs becoming that large. Maybe it will work tomorrow as it wasn't ever rotated yet ;)
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #6 on: December 17, 2018, 05:44:45 pm »
Hi @fabian, you was rigth, the nginx rotate the logs.
But, even with log rotation, the interface crashes when try load the log file.

About the tail -f solution, I think that can be a good choice, even with a big file the browser/php does not crash.

Like this

https://github.com/pfsense/pfsense-packages/blob/903855b9b2d726f09594f6caccad8015b6063ca8/config/squid3/34/squid_monitor.php#L48-L66
https://github.com/pfsense/pfsense-packages/blob/903855b9b2d726f09594f6caccad8015b6063ca8/config/squid3/34/squid_monitor_data.php
Logged

fabian

  • Hero Member
  • *****
  • Posts: 2436
  • Karma: 171
  • OPNsense Contributor (Language, VPN, Proxy, etc.)
    • View Profile
    • Personal Homepage
Re: Nginx Plugin :: Log Viewer
« Reply #7 on: December 17, 2018, 07:01:06 pm »
for now, I have replaced the backend call by a line by line read, which may fix the problem.

https://github.com/opnsense/plugins/pull/1075

opnsense-patch -c plugins 537da9cbe58d5291c5963df8e72fda5e7f770eba
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #8 on: December 17, 2018, 07:43:39 pm »
Patch applyed, but the browser still crash (tested in several browsers)  :(

# opnsense-patch -c plugins 537da9cbe58d5291c5963df8e72fda5e7f770eba
....
done
All patches have been applied successfully.  Have a nice day.


Logged

fabian

  • Hero Member
  • *****
  • Posts: 2436
  • Karma: 171
  • OPNsense Contributor (Language, VPN, Proxy, etc.)
    • View Profile
    • Personal Homepage
Re: Nginx Plugin :: Log Viewer
« Reply #9 on: December 17, 2018, 07:51:26 pm »
ok but the backend works?
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #10 on: December 17, 2018, 07:57:20 pm »
Yes, I deleted the error log and re-created it with  (currently has 10 lines) and I can see by GUI log viewer

Logged

fabian

  • Hero Member
  • *****
  • Posts: 2436
  • Karma: 171
  • OPNsense Contributor (Language, VPN, Proxy, etc.)
    • View Profile
    • Personal Homepage
Re: Nginx Plugin :: Log Viewer
« Reply #11 on: December 17, 2018, 08:04:05 pm »
the patch only reduces some memory use of PHP but it sends the same data. Since I do not have such a big log, it will be probably very hard for me to reproduce.

Do you know if the crash happens while rendering (creating a long table) or already when reading the response?
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #12 on: December 17, 2018, 09:36:36 pm »
Well, I don't know tell when the error happen. indeed I did not investigate it.
I will take a time for this..... and if you want,I can provide one of my log files to you reproduce the problem.

Logged

fabian

  • Hero Member
  • *****
  • Posts: 2436
  • Karma: 171
  • OPNsense Contributor (Language, VPN, Proxy, etc.)
    • View Profile
    • Personal Homepage
Re: Nginx Plugin :: Log Viewer
« Reply #13 on: December 17, 2018, 10:49:23 pm »
A log file would be good, please take care that you do not violate any data protection laws - so at least replace the IPs by :: and 0.0.0.0.
Logged

ccesario

  • Jr. Member
  • **
  • Posts: 79
  • Karma: 1
    • View Profile
Re: Nginx Plugin :: Log Viewer
« Reply #14 on: December 18, 2018, 12:39:03 pm »
Hi Fabian,
Could you please provide me an email to snd the file?
Best regards
Logged

  • Print
Pages: [1] 2
« previous next »
  • OPNsense Forum »
  • Archive »
  • 18.7 Legacy Series »
  • Nginx Plugin :: Log Viewer
 

OPNsense is an OSS project © Deciso B.V. 2015 - 2021 All rights reserved
  • SMF 2.0.17 | SMF © 2019, Simple Machines
    Privacy Policy
    | XHTML | RSS | WAP2