Reporting Database Stops Capturing Data

[jbhorner]

I received a notification today in Zenarmor that there was a new version of the engine. I upgraded. It seemed to work out okay--the status showed (and still shows) a green "Running" status for both the engine and database. Well, not so much.

A few hours later, I see that the reporting data stops to a period very close to when I udpated the engine. Stopping and restarting the engine doesn't have any impact. There is no option to restart the database, as I'm using a remote Elasticsearch database. There is no data showing. Period.

I recall this happened a bit of time ago when I was using the local database option as well. There was an upgrade, I installed the upgrade, and the data stopped. My only recourse at that time was to start with new databases. So much for any historical data.

Is this happening to anyone else? My hunch is that the process to "talk" to the database just stops, but that is conjecture. I wish there was a way to restart the database service--but no dice for a remote database connection. The remote database is health, showing green for all indicies. This is not a remote database problem. It feels like it's trying to use a previously established connection that is no longer valid.

This product seems so unfinished to me. Little nagging problems that appear. Example: After 1.16 was released, every IPv6 address tied to one device was showing up as another device. It's hard to imagine that some level of QA wouldn't have caught that. I had many back-and-forth discussions about policies with them about a month ago. It was fruitless. First they told me it was an IPv6 issue. (I was using MAC addresses to slot devices between different policies...I have no idea how they connected those dots.)

To their credit, their support team is responsive and friendly. But, their only recourse seems to be to ultimately reqeuest a remote desktop troubleshooting session. I've posted about another issue related to many reports not displaying data when using a remote database. It's easily reproducible. But they want a remote desktop session. This seems reasonable if they cannot reproduce the issue. (Not really--they should have sufficient logging and other tools to mitigate the need for remote desktop sessions.)

Anyway...my ultimate goal with this thread is to see if others are experiencing the reporting database just stopping at a point for no apparent reason. (Even though it claims it is running.)

[IHK]

We are trying to reproduce the problem. What is your ElasticSearch version?

[SuperMiguel]

Im running into the exact issue, i just installed fresh elastic 8.8 since i was getting error message about Zenarmor not supporting higher than that.

In my case i do see data in my remote elastic search instance, but the Zenarmor UI shows nothing, and the  Reporting Database Status is Stopped. Im not sure how to enable it.

[almodovaris]

I do have Zenarmor working with Elasticsearch 8.12.2. However, not on OPNsense, but on Debian 12.

[alex_62450]

Hi everyone,

I have been using OPNsense for sometime but started to use ZenArmor recently.

OPNsense runs on a computer and here is the current config / version :
Version    24.1.5_3    
Architecture    amd64    
Commit    0d4f3e17b

Initially, ZenArmor has been working fine until a recent upgrade : at the moment, the dashboard still shows some data and the traffic graph (throughput) displays live download/upload curbs but no more data in reports & in live sessions : there is only a mention "no data to display".

What would you please recommend ? 

Many thanks.

[rubeelam]

While it's difficult to diagnose without more information, such as logs or system configurations, it's possible that the upgrade process could have disrupted the connection between the engine and the remote Elasticsearch database, as you speculated. If the issue persists, it's essential to ensure that the CRM data enrichment process is functioning correctly. Since you mentioned that restarting the engine didn't resolve the issue, and the remote database appears healthy, it might be worth reaching out to Zenarmor's support team again for further assistance with troubleshooting the CRM data enrichment process.

[sy]

Hi,

Your index files could be malformed. Please run Reset Reporting Database in Settings - Reporting & Data - Database - Reset Reporting Database. This will remove all index files then re-create them.