Elasticsearch service fails to start on reboot of firewall

[richaras]

This was originally posted under OPNsense 25.7...but looks like I should have posted it here instead :-)

After upgrading to OPNsense 25.7, the Elasticsearch service does NOT restart after a firewall reboot.  I can fix it and get it to start by going to services>Elasticsearch>General and click on the SAVE button. The enabled box is already checked, and the service fails to start until I clink on SAVE.   Once I click SAVE, the service starts up and Zenarmor works great.

I am using the remote Elasticsearch DB with the os-elasticsearch7-maxit plugin from the mimugmail repo.  I have uninstalled / reinstalled the DB and have the same result, the Elasticsearc service does not start and trying to manually start the service fails until I click the SAVE button.

I'v read that clicking the SAVE button rewrites the config file, allowing the service to start...but my guess is the file is NOT being saved?

[sy]

Hi,

Have you checked the Elasticsearch log file for any errors? It should be located at /var/log/elasticsearch/elasticsearch.log.

[richaras]

I restarted the firewall to generate the error again.  The elasticsearch service failed to start...and as before, I click on SAVE for the elasticsearch service and it starts with no issues.

I grabbed text from both sides of the error to include the manual start of the service.

2025-09-10T11:06:25-08:00    Informational    configd.py    message 038dad05-ca24-4c00-8e6c-21593a4db120 ['os-elasticsearch7-maxit'] returned OK
2025-09-10T11:06:25-08:00    Notice    configd.py    [038dad05-ca24-4c00-8e6c-21593a4db120] Installing firmware package os-elasticsearch7-maxit
2025-09-10T11:05:33-08:00    Notice    configd.py    [a95f98ef-239d-42e1-81cd-5d5124d3021b] Querying os-elasticsearch7-maxit package details
2025-09-10T11:03:04-08:00    Informational    configd.py    message 0aeb38b6-b85b-4c91-8083-484f20ad0bba ['os-elasticsearch7-maxit'] returned OK
2025-09-10T11:03:04-08:00    Notice    configd.py    [0aeb38b6-b85b-4c91-8083-484f20ad0bba] Remove firmware package os-elasticsearch7-maxit
2025-09-10T11:01:56-08:00    Debug    configd.py      OPNsense/Elasticsearch generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:01:56-08:00    Notice    configd.py    generate template container OPNsense/Elasticsearch
2025-09-10T11:01:56-08:00    Notice    configd.py    [73328207-bb86-4856-82b9-16fdedd79a59] generate template OPNsense/Elasticsearch
2025-09-10T11:01:56-08:00    Notice    configd.py    [9ca3f9be-4413-4126-9608-a8169f7c7ee2] stopping Elasticsearch
2025-09-10T11:01:51-08:00    Notice    configd.py    [469ecf3e-46f0-4689-bf19-6b98031e0352] request Elasticsearch status
2025-09-10T11:01:16-08:00    Notice    configd.py    [d2a0f271-2ac5-461d-9491-04156c8839e0] request Elasticsearch status
2025-09-10T11:00:49-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:49-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:49-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:48-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:48-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:48-08:00    Error    configd.py    [e03fa892-b455-4ecf-8ae9-ba7d6de572f4] Script action failed with Command '/usr/local/zenarmor/scripts/installers/elasticsearch/create_indices.py '' ''' returned non-zero exit status 5. at Traceback (most recent call last):  File "/usr/local/opnsense/service/modules/actions/script_output.py", line 78, in execute    subprocess.run(script_command, env=self.config_environment, shell=True,  File "/usr/local/lib/python3.11/subprocess.py", line 571, in run    raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '/usr/local/zenarmor/scripts/installers/elasticsearch/create_indices.py '' ''' returned non-zero exit status 5.
2025-09-10T11:00:48-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:48-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:47-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:47-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:28-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:26-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:25-08:00    Debug    configd.py      OPNsense/Zenarmor generated //etc/rc.conf.d/elasticsearch
2025-09-10T11:00:00-08:00    Notice    configd.py    [24c03cea-085a-4335-af13-c91130b61710] checked remote elasticsearch
2025-09-10T10:56:45-08:00    Notice    configd.py    [f874d17d-f0d4-4142-970f-2870dafac254] request Elasticsearch status
2025-09-10T10:55:00-08:00    Notice    configd.py    [82e79511-44fd-46d3-9df5-9832832c8b09] checked remote elasticsearch
2025-09-10T10:50:00-08:00    Notice    configd.py    [fb95610c-b5ab-4542-8e20-17007ea874c9] checked remote elasticsearch
2025-09-

[sy]

Hi,

This is the error due to Zenarmor can not connect to Elasticsearch. If you use Elasticsearch on OPNsense, why you don't install it from Zenarmor repository?