NTOPNG will not start

[lshantz]

I have been using NTOPNG for awhile now and I'm not sure if the update to 20.7 borked it, or something else. The Redis service has started, but the ntopng service will not start. I've removed both redis and ntopng and started over, but something is corrupt. Even though I delete the redis program, when I re-install it, my old settings are still there, so it looks like I'll have to delete something from bash. Anybody know how?

[mimugmail]

Can you check the logs when restarting ntopng?
The saved settings stay also when removing, no problem

[lshantz]

I do  not know where the NTOPNG logs are kept. I looked at the FAQ and they say I can export the logs using journalctl, but that is not apparently native to opnsense. Or I don't know where that is either. Here is the tail of the Redis log:
38061:M 20 Aug 21:27:14.335 # User requested shutdown...
38061:M 20 Aug 21:27:14.336 # Redis is now ready to exit, bye bye...
64080:C 20 Aug 21:27:14.853 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
64080:C 20 Aug 21:27:14.853 # Redis version=4.0.14, bits=64, commit=00000000, modified=0, pid=64080, just started
64080:C 20 Aug 21:27:14.854 # Configuration loaded
17626:M 20 Aug 21:27:14.860 # Server initialized

I have found some locations for ntopng, but no ntopng.log

/etc/rc.conf.d/ntopng
/usr/local/opnsense/service/templates/OPNsense/Ntopng/ntopng
/usr/local/opnsense/version/ntopng
/usr/local/etc/rc.d/ntopng
/usr/local/share/ntopng
/usr/local/bin/ntopng
/var/mail/ntopng
/var/run/ntopng
/var/db/ntopng

I've read somewhere awhile back that the /var/db/ntopng/.lock file can cause it not to load. I have removed it, but that does not help. There is nothing else in the db directory

[mimugmail]

Ah sorry, I mixed it with netdata. Can you just check System : Disgnostics: Logs : General

[lshantz]

I'm sure based on the reply, you mean the gui based logs?

I cleared all the logs and then attempted to start NTOPNG.

2020-08-21T14:26:08   configd.py[21782]   [75b9fc22-56c7-494b-8a7c-16b474b73305] Show log
2020-08-21T14:26:01   configd.py[21782]   [b3585851-ab74-4e52-8c29-c45005487e26] Show log
2020-08-21T14:25:57   configd.py[21782]   [d338a6ff-321d-44cb-8195-98a1cf096398] request pfctl byte/packet counters
2020-08-21T14:25:51   configd.py[21782]   [14b509bb-f5ac-4d08-a55e-dc82b1566512] Show log
2020-08-21T14:25:49   configd.py[21782]   [877e266c-7cb3-46f8-8579-ba4e5e64b0e4] request pfctl byte/packet counters
2020-08-21T14:25:47   configd.py[21782]   [be69ae96-14b7-4087-a5fa-84d4f121d07a] returned exit status 1
2020-08-21T14:25:42   configd.py[21782]   [92369845-12d6-4443-87e7-1527abec220d] request pfctl byte/packet counters
2020-08-21T14:25:35   configd.py[21782]   [7b50e02d-e7b7-43bf-a43c-7889cba2d80b] request pfctl byte/packet counters
2020-08-21T14:25:31   configd.py[21782]   [be69ae96-14b7-4087-a5fa-84d4f121d07a] starting ntopng
2020-08-21T14:25:28   configd.py[21782]   [f4caead2-69b5-426b-a376-f46da5ed1736] request pfctl byte/packet counters
2020-08-21T14:25:21   configd.py[21782]   [e4d5882c-6f6a-4586-8479-e1d82565b612] Show log
2020-08-21T14:25:19   configd.py[21782]   [897437bb-d886-4ab6-816b-77fb3de56258] request pfctl byte/packet counters
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-lockout.conf
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-local.conf
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/syslog-ng-destinations.conf
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf.d/legacy.conf
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //usr/local/etc/syslog-ng.conf
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //etc/newsyslog.conf
2020-08-21T14:25:19   configd.py[21782]   OPNsense/Syslog generated //etc/rc.conf.d/syslog_ng
2020-08-21T14:25:19   configd.py[21782]   generate template container OPNsense/Syslog


This log seems to indicate that the old config is trying to use a credential. I had tried to clear that from the bash prompt.

020-08-21T14:25:47   root[55871]   /usr/local/etc/rc.d/ntopng: WARNING: failed to start ntopng
2020-08-21T14:25:47   ntopng[90139]   [Redis.cpp:150] ERROR: to specify a redis server other than the default
2020-08-21T14:25:47   ntopng[90139]   [Redis.cpp:149] ERROR: Please start it and try again or use -r
2020-08-21T14:25:47   ntopng[90139]   [Redis.cpp:148] ERROR: ntopng requires redis server to be up and running
2020-08-21T14:25:46   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:45   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:44   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:43   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:42   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:41   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:40   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:39   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:38   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:37   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:36   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.
2020-08-21T14:25:35   ntopng[90139]   [Redis.cpp:119] ERROR: NOAUTH Authentication required.

SO, the Redis server IS up and running. Is there some way to fire off NTOPNG from the bash and maybe see what is wrong there? There are no options for authentication in the png menu.

[lshantz]

Go figure. I deleted the password in redis and it came up!  I can't log in now, but I suspect it is the error message above. I'll go looking at FAQ's and figure it out. I added the pw back, but still no joy. I will report back if I figure it out.

Later: I can not log into the web page, but at least I got the service loaded. Somehow, I have managed to unsync redis and ntopng. In redis, if I leave the password in, then I get: Setting local networks to 127.0.0.0/8
21/Aug/2020 16:00:53 [Redis.cpp:119] ERROR: NOAUTH Authentication required.

if I blank out the password, then it loads fine. However, then I can't login in the GUI on port 3001. It just pops back to the prompt. Too many tries, and I get locked out for 5 minutes. So redis is NOT expecting a pw, but ntopng thinks it should have a password. ?

[lshantz]

I have finally gotten to the bottom of this. Hopefully it will help someone else. Redis had a password embedded into the config  file! Somehow in all my messing around and getting bad advise I must have done a command line issue. So even though you have a password set in redis config in the gui, it has no bearing on the redis server. I assume this is a bug, or maybe it should be beefed up. In any event here we go:

https://curiousviral.com/error-noauth-authentication-required/

Do a 'find / -name for redis.conf'. Then go edit it and remark out the password line. Restart the process and you should finally be good to go.

[lshantz]

I wonder if I should start a new thread. This is at least the 2nd time a system upgrade/update has borked NTOPNG. It has been running fine since August. I just did an update last night and sure enough, NTOPNG is borked again. GAH.

[lshantz]

Oh, and I found out where the NTOPNG log file is kept, now that it has been running.
/var/db/ntopng

So looking at the ntopng log, there is no activity in my attempts to restart it. Redis, shows it is starting and waiting for a connect. NTOPNG drops a .lock file and does not start.

Hopefully someone has an idea. Otherwise, I'll report back when I figure it out.

Later: We are back to [Redis.cpp:119] ERROR: NOAUTH Authentication required.

This makes no sense. NOTHING was done, except do an upgrade! It seems like somehow it pulled old data on the upgrade.

Okay found it: In Services/Redis/restrictions. There is a location for Server Password. As recommended, I had one in there. I deleted, and up it came. So somehow this last 2 updates, borks this. It should be able to reproduce if anyone cares to fix it.

[lshantz]

For some reason, now I can't have a password in the services/redis/restrictions/server password any longer. ?? No idea why. I know it is supposedly insecure, but until someone comes up with an answer, I guess that is how it has to be.

I just did a restore from a previous backup and it failed again. I came back here to recall what I did. Deleting the password and up it came. 

[mimugmail]

For some reason, now I can't have a password in the services/redis/restrictions/server password any longer. ?? No idea why. I know it is supposedly insecure, but until someone comes up with an answer, I guess that is how it has to be.

I just did a restore from a previous backup and it failed again. I came back here to recall what I did. Deleting the password and up it came.

Dont use a password for redis ...

[lshantz]

Something changed.
1. it is insecure to have no password.
2. It has worked this way for quite a long time now.
3. Something changed and now it has no password to work.

NOT secure! Ironic from a security device. I have an idea, FIX IT.

[mimugmail]

I have an even more better idea. Make it work with Password or dont use it If you think it's insecure.

Redis is a cache .. like a flatfile. Is netflow via sqlite password protected? No. Is the database for vouchers encrypted? No.

If you have the time to fix it, happy to review your patches

[lshantz]

So here I am again. A sector went bad on my SSD. Total Firewall fail. Would not boot. I reformatted, re-installed from scratch. I then restored from the latest backup, which was very current. Apparently not all things are backed up. Numerous services are not loading including NTOPNG. Same issue. I could not log in. I had to issue command line commands to reset the password, even though I was not aware a REDIS password was in use. I am fairly sure it was the NTOPNG password, and it had to be reset to default to regain access to it. ??