ET Pro Telemetry widget fail to load/empty with 24.7.12

[ivwang]

Hi all,

After 24.7.12 upgrade, it looks like the ET Pro Telemetry widget shows either empty box or "failed to load widget".

From the CLI, sensor_info.py *sometimes* works, and when it works it takes several seconds to complete, though it reports the sensor is still ACTIVE, while other times the script failed with exception stating remote end closed connection, like below:

--------
  File "/usr/local/lib/python3.11/http/client.py", line 294, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
--------

Anyone also seeing this?

Thanks a lot.

[RamSense]

yes here the same, see also here: https://forum.opnsense.org/index.php?topic=45112.0
and here: https://forum.opnsense.org/index.php?topic=40751.0

[RES217AIII]

I have the same problem
Is this possibly the answer to the problem?

https://community.emergingthreats.net/t/etpro-telemetry-edition/2355

"For sensors opting-in to sending Proofpoint/ET telemetry so they can receive ETPRO telemetry edition those sensors must have sent event telemetry back to Proofpoint/ET within the last 5 days.

Sensors may go dormant during that period (no heartbeat sent in the last day) and still receive ETPRO Telemetry Edition, but if no events are received for 5 days the Telemetry Edition rule delivery will be disabled and that sensor will simply receive that day's ET Open rules.

That disabling will transition back to active delivery upon resumption of heartbeat and telemetry delivery back to Proofpoint/ET.

Sensors are reviewed as to state every 24 hours."

[RamSense]

I have installed a new opnsense box and ordered a new token for ET Pro Telementry, and the same result with the widget.
So maybe there is something with the widget or with opnsense not sending the heartbeat??

[irrenarzt]

Same issue, but I was starting to have problems before 24.7.12. The token was expiring after approximately 5 days despite consistent heartbeats. I contacted ET Labs and they said they've received multiple reports of this and were looking into it.

Something has further degraded though... just like you guys, updating the token isn't fixing the issue anymore. I can't update ET Pro rules, the widget doesn't work, and I'm getting the same error in my logs as above. That said, I'm confident it's not an OPNsense issue.

[sbaran50]

Exactly the same issue here, STATUS_PY says its ACTIVE.
But mine struggles to send python3 send_heartbeat.py.
Its doing nothing for 10 minutes, last HB was sent yesterday, after i have completely reinstalled the FW from scratch.

[jglatz]

I am having the same issue on 24.10.1, been flaky for a month or so. I've had to order new tokens to get it working. Starting in the last week, I am seeing the exact same thing. Widget won't load, invoking the sensor_info.py command takes a very long time when it works. Most of the time it will fail

[klamath]

24.10.1-amd64 same issue:

  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 682, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[Monviech (Cedrik)]

The issue is tracked here:

https://github.com/opnsense/plugins/issues/4486

We will respond there with new info when we have it.

[corran22]

I have the same problem
Is this possibly the answer to the problem?

https://community.emergingthreats.net/t/etpro-telemetry-edition/2355

"For sensors opting-in to sending Proofpoint/ET telemetry so they can receive ETPRO telemetry edition those sensors must have sent event telemetry back to Proofpoint/ET within the last 5 days.

Sensors may go dormant during that period (no heartbeat sent in the last day) and still receive ETPRO Telemetry Edition, but if no events are received for 5 days the Telemetry Edition rule delivery will be disabled and that sensor will simply receive that day's ET Open rules.

That disabling will transition back to active delivery upon resumption of heartbeat and telemetry delivery back to Proofpoint/ET.

Sensors are reviewed as to state every 24 hours."

Greetings - we've modified the token code to re-enable sensors which had been disabled in this period as well as open up the window that's examined to determine whether a sensor is still sending us data (or not). Apologies for the disruption. We'll get some documentation out clarifying our position on telemetry reception and periodicy soon.--ET Team

[RayonRa]

Hi,
i still have the issue.
https://github.com/opnsense/plugins/issues/4486#issuecomment-2608435385

[corran22]

We're looking at the 502 errors - this is not a sensor disabling issue.  Resource-wise our internal monitoring shows the server healthy and responsive.  Are you still having this problem?

[RayonRa]

We're looking at the 502 errors - this is not a sensor disabling issue.  Resource-wise our internal monitoring shows the server healthy and responsive.  Are you still having this problem?

Yes, i still have the problem.

2025-01-25T10:50:52   Error   send_telemetry.py   unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 502)   
2025-01-25T10:49:11   Error   send_telemetry.py   unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 502)   
2025-01-25T10:48:24   Error   send_telemetry.py   unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 502)


I tried a rule updater:

2025-01-25T11:09:49   Error   rule-updater.py   download failed for https://opnsense.emergingthreats.net/api/v1/ruleset/version (http_code: 502)   
2025-01-25T11:09:48   Error   rule-updater.py   download failed for https://opnsense.emergingthreats.net/api/v1/ruleset/version (http_code: 502)   
2025-01-25T11:09:48   Error   rule-updater.py   download failed for https://opnsense.emergingthreats.net/api/v1/ruleset/engine/suricata/5 (http_code: 502)   
2025-01-25T11:09:47   Error   rule-updater.py   download failed for https://opnsense.emergingthreats.net/api/v1/ruleset/version (http_code: 502)   
2025-01-25T11:09:47   Error   rule-updater.py   download failed for https://opnsense.emergingthreats.net/api/v1/ruleset/engine/suricata/5 (http_code: 502)

[RayonRa]

After a reboot the 502 is gone.
I'm able to download the rules.
BUT (see screenshot)
And i can't see send_telemetry.py work.