Zenarmor 1.14: External Elastic database - no data available in reports

[serbans]

After updating to 1.14, there is no report available/no data available in the dashboard or in the reporting, live sessions, activity explorer, etc.
Reboots performed.
Accessing the firewall via the ip address, with FQDN all fields are "network error".

Running licensed Home version, OPNSense 23.7

[sy]

Hi,

The issue is fixed and a bugfix release will be shipped today.

[stomek]

Good evening everyone,

have installed the bug fix relase and the 2 reported issues seems to be sorted properly.
Until now furhter anomalies could have been found - software smoothly in the entire GUI menu tree.

Have a nice evening,
Stefan

[serbans]

For me the patch has not corrected the fact that with an external elasticsearch database, I am still unable to see any traffic reports.

Status:
- in the settings/configuration page
    - reporting database  - elasticsearch (remote) - cannot be changed either when the engine is running or stopped
     - the field "remote url" does not contain the port information, adding it it says "saved" nd after a page reload it is gone

- in the settings data management page
     - stream reporting data to elasticsearch - I have configured the url and enabled it.

- in the dashboard page
     - regardless of the setting of the stream reporting data to elasticsearch - the setting Reporting database shows "elasticsearch". If I click on start, it shows someting starting and there is an elasticsearch locally running on the firewall.

Next step - removing the module completely and installing again.

UPDATE - reinstalled ZenArmor, still same issue, it seems that it tries a local elasticsearch instance for reports and such even if it is configured with an external one

I will try and open a ticket with Zenarmor as well.

[badbroccoli]

For me the patch has not corrected the fact that with an external elasticsearch database, I am still unable to see any traffic reports.

Status:
- in the settings/configuration page
    - reporting database  - elasticsearch (remote) - cannot be changed either when the engine is running or stopped
     - the field "remote url" does not contain the port information, adding it it says "saved" nd after a page reload it is gone

- in the settings data management page
     - stream reporting data to elasticsearch - I have configured the url and enabled it.

- in the dashboard page
     - regardless of the setting of the stream reporting data to elasticsearch - the setting Reporting database shows "elasticsearch". If I click on start, it shows someting starting and there is an elasticsearch locally running on the firewall.

Next step - removing the module completely and installing again.

I will try and open a ticket with Zenarmor as well.

I am having the same issues.

edit: ZA support claims a known issue w/ remote ES and another hotfix will be released today.

[dotlike]

For me the hotfix solved the problem to access the Zenarmor-GUI.
The problem is that I get a lot of false positives (for NTP, DNS,...). I have to set Zenarmor to bypass mode to get online again.
I have tried to reset Zenarmor to factory defaults without success.

The worst upgrade so far 

[serbans]

Update:

After removing all indices and data views relted to ZA from ES, managed at the third install to have the environment up and running.
Still having some issues with the reporting ("ZA detected 8 and blocked 0 potentially harmful activities" ) but the ES part seems to be resolved.

[dotlike]

Update 1.14.2 fixed my issues 

[sy]

Hi,

@Serbans, you can see the caught Threats in Reports - Threats tab. This means that Zenarmor caught but there is no rules to block them. If you set as blocked the caught category(ies) then you will see there them as blocked.

[diba]

Hi,

i have still the problems with reports etc. when i use an external opensearch db after the update to 1.14.2.

Data was written to the DB but no data is showen in the reüports

[pwalczak]

More information from my setup as well.

I have configured an external ES DB after upgrading Zenarmor to 1.14.2. It basically works but some reports don't display data.

These reports are:

• Egress New Connections by App Over Time
• Egress New Connections by Source Over Time
• New Connections & Unique Remote Hosts
• Unique Local Hosts
• Facts/Connections is set to NaN
• Facts/unique Local Devices is set to 0
• HTTP Transactions by Source Over Time
• Top Egress Users
• Top Ingress Users
• Top OS
• Top Session Creators Over Time
• Top Servers Over Time

I would expect to see data in at least some of them.
One more factor that can make a difference is the fact I use custom lifecycle policy and custom index names in ES. However I've made sure all mappings are defined exactly the same way as indices created by Zenarmor during the installation process.

[tokar86a]

This still seems to be a problem whit this? I have ended up whit the same error. No data to display.

[sy]

Hi all,

1.15 has fixes for the report charts. Can you try the reports after the update?

[tokar86a]

Hi all,

1.15 has fixes for the report charts. Can you try the reports after the update?

Still has the same problem.

[idpitt]

Similar issues here. Running 1.15.1 on OPNSense 23.7.6 with an external elasticsearch DB. I see indexes being populated with data and some reports display info in the zenarmor GUI.     Unique local hosts for example is blank.  Activity Explorer is totally blank.  A 'reset indexes' attempt returns a failed to connect yet deleting data works. 

When I was running the elastic search locally, the reports were working fine.  I wanted to offload the database to an existing ELK server.