Periodic Speedtest

[FullyBorked]

I'm not sure when it stopped working but my periodic speedtest cron job doesn't seem to be working.  At least I don't see any tests logged and the widget doesn't show any data.  The cron job seems pretty straight forward but doesn't seem to work.  I know you need to set a parameter for the server id, I'm just using the id digits, does it also need brackets or parenthesis?  Not fully sure how to troubleshoot it. 

[doverton12]

This may sound strange, but open a shell via ssh and check the speedtest csv file (/usr/local/opnsense/scripts/OPNsense/speedtest/speedtest.csv).  Mine had the header row repeated. I deleted the 2nd copy of headers and it all started working.

Thanks

David

[FullyBorked]

Wow good job finding that, I'd completely given up on solving it.  Looks like this is your first post too, so very thankful you took the time to make an account to help me out.  Posted a screenshot of the duplicate header.

[Pyrocumulous]

@doverton12 good find, deleting the duplicate header solved the undefined statistics for me as well.

[asansi]

Yup, same here, thanks for that!  (double header - fixed with removing one line)

[zz00mm]

Rebuild an HA Cluster configuration. speedtest info within the GUI showed nothing. Checked the speedtest.csv file it didn't have a duplicate entry, nano did say it was converting from a DOS file. Nothing looked wrong, so I renamed speedtest.csv to speedtest.csv.org and ran "python3 opn_speedtest.py" which recreated the speedtest.csv file and populated it with speed output information. Information now shows in the widget in the GUI.

[beneix]

My nightly cron job also stopped working. I did not have a duplicate header in speedtest.csv but decided to try @zz00mm's approach, so I deleted speedtest.csv and ran "python3 opn_speedtest.py". This did not perform a speedtest, but did create a new speedtest.csv with only a header in. It also gave the following errors:

Code Select Expand

Traceback (most recent call last):
  File "/usr/local/opnsense/scripts/OPNsense/speedtest/opn_speedtest.py", line 183, in <module>
    result = json.loads(subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, check=True).stdout.decode('utf-8'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Any suggestions what these errors mean?

[timtrace]

I ran

Code Select Expand

sudo -i
cd /usr/local/opnsense/scripts/OPNsense/speedtest/opn_speedtest.py
python3 opn_speedtest.py


I got the same error as beneix, then I remembered to rm speedtest.csv, now the command works.

Jury is still out on whether anything magically fixed the cron job tho.

[doktornotor]

No errors... No duplicate CSV header.

Code Select Expand


# python3 opn_speedtest.py
{"timestamp": "2024-08-16T05:12:11Z", "clientip": "109.xxx.xxx.xxx", "serverid": "30315", "servername": "Stormmedia, Aleksandrow Lodzki", "country": "Poland", "latency": 45.69, "download": 167.18, "upload": 39.76, "link": "http://www.speedtest.net/result/16630202214"}


- No idea why speedtest thinks I am located in Poland. This happens periodically with boxes located in multiple countries, none of which happens to be Poland, and goes away randomly after a couple of days.
- Randomly get just 403 errors when trying to run speedtest-cli from shell.

The above happens even on boxes not running OPNsense at all.

As for OPNsense - according to the GUI log, the cronjob has been working very much randomly every couple of days. This only happens on boxes connected to major ISPs. No problems with some obscure ones with a couple of thousands of customers.

Conclusion: Ookla speedtest has broken geolocation plus some "FUP" applied per ASN or whatever. Not exactly reliable or usable. Moving on, debugging this is a waste of time, the problem is with speedtest.net and not the package.

[arut]

i checked the result befor jsonconvert and there is a missing ","

{
   "type":"log",
   "timestamp":"2025-02-02T20:09:17Z",
   "message":"Error:

• Timeout occurred in connect.",

   "level":"error"
}\n{                                                         <-- her shoulbd be a ","
   "type":"result",
   "timestamp":"2025-02-02T20:09:28Z",
   "ping":{"jitter":0.081,
   "latency":3.562,
   "low":3.424,
   "high":3.616
},
"download":{

[arut]

her is also some copy paste mistake

   # parameter q or list - returning the list of servers available
    if arg=='t' or arg=='list':

[arut]

i removed the error output from result
result = json.loads(subprocess.run(cmd, stdout=subprocess.PIPE, check=True).stdout.decode('utf-8'))

now all is working fine