OPNsense Forum

Archive => 24.1, 24.4 Legacy Series => Topic started by: lshantz on February 14, 2024, 08:51:13 PM

Title: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 14, 2024, 08:51:13 PM
After the update REDIS no longer works. I watched on an attached monitor upgrading another box and saw where it said something about directory invalid or not found. So am I the only one? it happened on 2 different boxes.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on February 15, 2024, 10:15:29 AM
Did you try to reset the db?
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 15, 2024, 08:28:14 PM
I've not had time to do much. Fighting getting two upgraded systems happy. I did a reinstall, but didn't bother to look at the log yet to see if there were the same error messages. I would assume anybody that did the upgrade had the same issue. Apparently a bad assumption.  :P 

I will have to re-read the docs to see if that is all it needs and how to do it.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 21, 2024, 12:09:45 AM
So looking in /var/db/redis, it is empty!

I looked at the logs and got:
/var/log/system/system_20240220.log | grep redis
<13>1 2024-02-20T10:43:47-07:00 xxxxxxxxxxxxxx.com kernel - - [meta sequenceId="314"] <118>Starting redis.
<12>1 2024-02-20T10:43:47-07:00 xxxxxxxxx.com redis 97787 - [meta sequenceId="315"] WARNING: The TCP backlog setting of 511 cannot be enforced because kern.ipc.somaxconn is set to the lower value of 128.
<12>1 2024-02-20T10:43:47-07:00 xxxxxxx.com redis 97787 - [meta sequenceId="316"] Warning: Could not create server TCP listening socket xxxxxxx.8:6379: bind: Address already in use
<12>1 2024-02-20T10:43:47-07:00 xxxxxx.com redis 97787 - [meta sequenceId="317"] Failed listening on port 6379 (tcp), aborting.
<11>1 2024-02-20T10:44:29-07:00 xxx.com ntopng 20907 - [meta sequenceId="369"] 20/Feb/2024 10:44:29 [Redis.cpp:153] ERROR: ntopng requires redis server to be up and running

Some of this may be erroneous, since we did change from .1 to .8 trying to get CARP running. It failed prior to this. Hopefully this will make sense to someone. I can't find the ipc.somaxconn setting anywhere.

Any help appreciated.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 21, 2024, 12:36:24 AM
HA, I tried to reinstall again and I see there is a 24.1.2, so maybe that will fix it. Fingers crossed.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on February 21, 2024, 06:58:55 AM
This is the error:
Could not create server TCP listening socket xxxxxxx.8:6379: bind: Address already in use
<12>1 2024-02-20T10:43:47-07:00 xxxxxx.com redis 97787 - [meta sequenceId="317"] Failed listening on port 6379 (tcp), aborting.

The 511 is just a warning.
Go to CLI, sockstat -4 and search for the port and process
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 21, 2024, 06:46:44 PM
->Go to CLI, sockstat -4 and search for the port and process

Thanks for replying. Good to know that is a simple warning, but I did stop the warning in turneables section.

Okay, but it doesn't matter what port you set it to, you still get the same error. I did do as you suggested and it is not listed.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on February 21, 2024, 07:28:06 PM
And you hit the "reset db" in redis, correct? Can you post a screenshot or redis config page?
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 27, 2024, 02:33:16 PM
I did the reset, but no change.
I attached a copy of the setup page.

Here is the log. It is still unhappy with the port, so that HAS to be somehow related, but like I said I changed ports and it was still not happy. ??

024-02-27T06:16:37-07:00   Warning   redis   Failed listening on port 6379 (tcp), aborting.   
2024-02-27T06:16:37-07:00   Warning   redis   Warning: Could not create server TCP listening socket 192.168.100.1:6379: bind: Address already in use   
2024-02-27T06:16:37-07:00   Notice   redis   Running mode=standalone, port=6379.   
2024-02-27T06:16:37-07:00   Notice   redis   monotonic clock: POSIX clock_gettime   
2024-02-27T06:16:37-07:00   Notice   redis-server   Configuration loaded   
2024-02-27T06:16:37-07:00   Notice   redis-server   Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=31815, just started   
2024-02-27T06:16:37-07:00   Notice   redis-server   oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on February 27, 2024, 05:53:50 PM
Can you download config.xml, remove the whole redis Container, restore, reboot, remove os-redis, install again and start without ANY modification beside enable checkbox
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 27, 2024, 06:07:11 PM
Sure, I'm willing to do that, but have two work from homes right now, so will try that tonight.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 27, 2024, 10:14:09 PM
Well I did as you suggest and deleted everything REDIS. Alas I got: Installation out of date. The update to opnsense-24.1.2_1 is required.

I've been holding off because my last two updates were a nightmare. Now I have to decide if I want to update and risk being down again, or take a chance.  :-\
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on February 28, 2024, 02:46:32 PM
If you type "pkg install os-redis" via CLI, you dont need to be on the latest version :)
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 28, 2024, 06:45:37 PM
DOH! I did not realize that. I guess I'm getting rusty at CLI and becoming a child of the GUI.

Okay, so I did completely delete everything related to REDIS. Then installed via CLI. I got the exact same error message! This is so weird. I did a netstat -l and there is no 6379. I even changed it to 6378 and it gives the same error.  I wonder if the Opnsense update has changed some environment and it is getting confused, although apparently few are having this issue.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on February 29, 2024, 07:35:58 AM
I did a fresh install on a clean system, latest updates, just installed redis, NO changes (also no listen oder password), on enabled and it works.

Interface : Diagnostics : Netstat : Sockets -> search for 6379 lists the open port with command redis-serv
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on February 29, 2024, 09:56:26 PM
I did it from CLI and not there. Did it as you mention, not there. I wonder if it is possible that somehow NTOPNG is somehow causing the issue. Hate to blow it out of the water to prove it. Guess I could rename the directory uninstall try again with just REDIS. It is a head scratcher for me. It does not matter what port I put it to, it always says the same thing, that was what lead me to believe it is a problem with the version of REDIS.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on March 09, 2024, 02:11:15 AM
So I got brave and updated to 24.1.3_1 and no issues, other than REDIS. It still does not come up. I was doing some searching and I have found something of interest. I Hope it is ok to paste a link for the whole story in case anyone wants to read it: https://stackoverflow.com/questions/7417232/redis-server-port-already-in-use?rq=4

The relevant  parts are: redis-cli ping, redis-cli shutdown, redis-server. Since I already knew there was no services running, I went straight to redis-server. What do you know, it came right up!!!
redis-server
45155:C 09 Mar 2024 00:23:39.950 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
45155:C 09 Mar 2024 00:23:39.950 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=45155, just started
45155:C 09 Mar 2024 00:23:39.950 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
45155:M 09 Mar 2024 00:23:39.956 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 7.2.4 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
(    '      ,       .-`  | `,    )     Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
|    `-._   `._    /     _.-'    |     PID: 45155
  `-._    `-._  `-./  _.-'    _.-'
|`-._`-._    `-.__.-'    _.-'_.-'|
|    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
|`-._`-._    `-.__.-'    _.-'_.-'|
|    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

45155:M 09 Mar 2024 00:23:39.963 # WARNING: The TCP backlog setting of 511 cannot be enforced because kern.ipc.somaxconn is set to the lower value of 128.
45155:M 09 Mar 2024 00:23:39.964 * Server initialized

Opnsense still says it is not running, but I can fire up NTOPNG.

So, I would argue, there is something amiss with how the GUI is implementing it. REDIS DOES work, just not from their GUI. Something in the update borked it.

Edit: as a work around I did redis-server &, and it is running fine for now. Still show red in the Services dashboard, hopefully an admin will see this and help figure out what has gone sideways.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on March 09, 2024, 02:31:23 AM
I'm just good enough to be dangerous to myself. I started poking around and there are 2 config files on the firewall.
/usr/local/opnsense/service/templates/OPNsense/Redis/redis.conf
/usr/local/etc/redis.conf


# Redis configuration file.
#
# Note that in order to read the configuration file, Redis must be
# started with the file path as first argument:
#
# ./redis-server /path/to/redis.conf

/usr/local/bin/redis-server

So my question is, which config file is in use, and is the gui properly doing the above? Obviously not, but how do I figure out the bread crumbs from here?
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: mimugmail on March 09, 2024, 09:38:04 AM
The first one is the template, picking the vlues from gui and creates the second one :)
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on March 09, 2024, 05:26:44 PM
Okay, I suspected as much. Didn't even look at the template one, but how does that help me solve the mystery?  :o
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: victor.diuk on March 27, 2024, 10:03:26 AM
Same issue. 24.1.4 - Redis and NtopNG Showed as "working services" but no access...
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: franco on March 27, 2024, 10:13:47 AM
Can we stop this thread please? Make your own with the required info attached.


Cheers,
Franco
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on March 28, 2024, 12:53:11 AM
Obviously someone finally paid attention and corrected what ever got borked. It is now working after the latest update. You can lock/close this thread now. It would be nice to get an acknowledgement of what was wrong and what was done to fix it in the patch though.
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: franco on March 28, 2024, 01:34:46 PM
From an engineering perspective no code changed in this area. I'm not sure what problem you were having.


Cheers,
Franco
Title: Re: 24.1.1 NTOPNG and REDIS no longer working
Post by: lshantz on March 28, 2024, 05:07:03 PM
 :o Really!! Dang... Well thanks for replying and keep up the good work.

At least I have documented a work around above for anyone else suffering from this weird issue. Oh, one thing.. it takes awhile to configure NTOPNG how I want it and it appears that this config data is stored in REDIS. I would very much like to capture/save this config data so when it does down again, I can quickly restore back to my setup. This is the 3rd or 4th this has happened over the last couple of years. Thanks in advance.