Elastic search won't start on 22.1

[sy]

Hi @Armin,

For the early updaters, It seems that the new repo is installing without any issue. But It should be uninstalled from Zenarmor GUI (Configuration - Uninstall). Otherwise, some files don't remove and occur error.

And @Armin has still an error after reboot. There was a problem with index files and resolved by resetting report files (Configuration - Reporting & Data - Reset Reporting).

[Mks]

Hi,

thanks, that did the trick for me!

br

[ArminF]

bad news..

issue came back after antoher reboot. Even worse. System resettet itself every 1-2 minutes.
was pretty hard to get it back to work.

Had to
service elasticsearch stop
rm -rf /etc/rc.conf.d/elasticsearch
rm -rf /usr/local/opnsense/service/templates/OPNsense/Sensei/rc.conf.d/elasticsearch
rm -rf /usr/local/etc/elasticsearch
rm -rf /usr/local/etc/rc.d/elasticsearch
rm -rf /usr/local/libexec/elasticsearch
rm -rf /usr/local/libexec/elasticsearch/bin/elasticsearch
rm -rf /usr/local/lib/elasticsearch
rm -rf /usr/local/lib/elasticsearch/bin/elasticsearch
rm -rf /usr/local/sensei/scripts/database/templates/rc.conf.d/elasticsearch
rm -rf /usr/local/sensei/scripts/updater/elasticsearch
rm -rf /usr/local/sensei/scripts/installers/elasticsearch
rm -rf /usr/local/datastore/elasticsearch
rm -rf /var/mail/elasticsearch
rm -rf /var/run/elasticsearch
rm -rf /var/db/elasticsearch
rm -rf /var/tmp/elasticsearch
rm -rf /var/log/elasticsearch

directly from cli. Right after the next reset it came back stable.
I am not sure what this is all about.
What i can say is that this is elasticsearch related.
I guess there is something left over which causes the trouble for elasticsearch.
So worse that it brings the system to reset all the time.
I can run the system stable with zenarmor and mongodb.

It might would be time to reinstall a 22 version and restore the backups.

[sy]

Hi Armin,

Is there any news on the issue after your support email? I think it is up now with Mongo DB. But it should be a system or hardware issue. When we checked it, the indexes were broken for a while.

[ArminF]

Hi Sy,

it is an elasticsearch issue which came with zenarmor at the time.
At the moment zenarmor runs with mongodb.

From system side of view i wrote that i might install a fresh build as the system was upgraded several times.
Just to strike out all old left over files of elasticsearch or zenarmor which cause the issue if reinstalled.

thanks for your help and support.
armin

[meazz1]

I got the same issue on 22.1 after I updated sunnyvalley to 1.2.1, os-sesi to 1.10.1.
So, I uninstalled and reinstalled zenarmor, while coiffuring ZI get the error  in hardware configuration "We could not locate ubench package, tried to install it, and installation did not work.".

Help pls.

[donatom3]

I got the same issue on 22.1 after I updated sunnyvalley to 1.2.1, os-sesi to 1.10.1.
So, I uninstalled and reinstalled zenarmor, while coiffuring ZI get the error  in hardware configuration "We could not locate ubench package, tried to install it, and installation did not work.".

Help pls.

I'm seeing the same thing wondering if it's possibly the repo.

[sy]

Hi,

Can you share the commands output;

pkg clean
pkg update
pkg search ubench

[meazz1]

Hi,

Can you share the commands output;

pkg clean
pkg update
pkg search ubench

I have reinstalled Zenarmor but did not configure again yet because of the ubench error I got yesterday.

pkg clean output
root@router:~ # pkg clean
The following package files will be deleted:
        /var/cache/pkg/os-sunnyvalley-1.2_1~c903d6c917.txz
        /var/cache/pkg/os-sensei-1.10.1~2e63a9cffd.txz
        /var/cache/pkg/os-sunnyvalley-1.2_1.txz
        /var/cache/pkg/os-sensei-updater-1.10.txz
        /var/cache/pkg/os-sensei-1.10.1.txz
        /var/cache/pkg/os-sensei-updater-1.10~9249ba3dac.txz
        /var/cache/pkg/ubench-0.32~bbd4f2ad21.txz
        /var/cache/pkg/ubench-0.32.txz
        /var/cache/pkg/os-sensei-agent-1.10.5~f5aa1cfce9.txz
        /var/cache/pkg/os-sensei-agent-1.10.5.txz
The cleanup will free 48 MiB

Proceed with cleaning the cache? [y/N]: y

oot@router:~ # pkg update
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
SunnyValley repository is up to date.
All repositories are up to date.

pkg search ubench
ubench-0.32                    Unix Benchmark Utility for CPU(s) and memory

By the way, I hit Y to pkg clean and cleaned out the pkgs.

[sy]

Hi,

What if you install ubench manually?

pkg install -fy ubench

[meazz1]

Hi,

What if you install ubench manually?

pkg install -fy ubench

When I try to install ubench manually it gave the beloww out put, but I think after running pkg clean something good happened. The configuration went on and completed.
pkg install -y ubench
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
SunnyValley repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The most recent versions of packages are already installed


Thank you for the pkg clean suggestion, I believe that worked.

[donatom3]

I just waited and I can run through no issues today.

[ArminF]

Ok, i tried it now with
Elasticsearch
MongoDB
SQLite

All installations start failing after a while.
I guess it is time to reinstall OPNSense.

SQLite worked the longest time. 1-2 month. While the other two failed after days/weeks.
Reduce reporting to one day. Resetted, reinstalled etc...

[ArminF]

Just like Ying & Yang - always something good on something bad.

this morning i reinstalled the firewall completely.
Was pretty straight forward with the configuration import.
Took 30 minutes..

So for now fresh system and Zenarmor is installed with the default
Elasticsearch
100 Devices
5 Days reporting data

As ADS and Tracker can be blocked by Zen i removed DNSBL.
And added all the o365 ranges for better traffic.

Lets see how it runs from now on.

[ArminF]

Well, right after one day elasticsearch died again....

Reinstalled yesterday
BUT restored configuration. Not sure if this can cause an error.

Zenarmor packed engine is running.

Could this be a hardware issue?
Am i somehow able to gather more details, logs etc?