OPNsense API & Home Assistant broken -- "gateway API dashboard" related?

[Grossartig]

I upgraded from 20.7.8 to 21.1.1 last night. Ever since, my Home Assistant instance is no longer able to communicate with OPNsense via the API. The error I see in Home Assistant is shown below.

For good measure, after first seeing the issue this morning, I generated a new key/secret pair and changed the HA configuration accordingly, just to verify that the old key/secret weren't the issue, but the problem persists. As you can see from below errors, my OPNsense WebUI is running on port 4444, as the Nginx plugin is already occupying 443 here.

Happy to reach out to the Home Assistant forums as well, but as I said, all was working until the upgrade to 21.1.x last night. Here is more information on how the integration between HA & OPNsense works: https://www.home-assistant.io/integrations/opnsense/

Code: [Select]

Log Details (ERROR)
Logger: homeassistant.setup
Source: components/opnsense/__init__.py:53
First occurred: 8:13:42 AM (1 occurrences)
Last logged: 8:13:42 AM

Error during setup of component opnsense
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 531, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='192.168.5.1', port=4444): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 213, in _async_setup_component
    result = await task
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/opnsense/__init__.py", line 53, in setup
    interfaces_client.get_arp()
  File "/usr/local/lib/python3.8/site-packages/pyopnsense/diagnostics.py", line 52, in get_arp
    return self._get('diagnostics/interface/getArp')
  File "/usr/local/lib/python3.8/site-packages/pyopnsense/client.py", line 51, in _get
    response = requests.get(req_url, verify=self.verify_cert,
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='192.168.5.1', port=4444): Read timed out. (read timeout=5)


[Grossartig]

Is this perhaps related to the following change in the changelog of OPNsense 21.1?

system: removed unused gateway API dashboard feed

[Grossartig]

Just a friendly nudge on this. Could someone confirm that the API-related change in 21.1 is the likely cause of this?

[franco]

I have no knowledge of what this code uses or not so I cannot answer the question.

And, yes, sometimes we remove obsoleted API pieces in major releases where replacements exist.


Cheers,
Franco

[Grossartig]

Understood -- thank you for the response

[QBANIN]

I have the same issue but IMO this is some kind of API issue in 21.1.* . I can't even access it using curl method https://docs.opnsense.org/development/how-tos/api.html#using-curl

[franco]

I have the same issue but IMO this is some kind of API issue in 21.1.* . I can't even access it using curl method https://docs.opnsense.org/development/how-tos/api.html#using-curl

Does your GUI work? See, API not broken...


Cheers,
Franco

[QBANIN]

I have the same issue but IMO this is some kind of API issue in 21.1.* . I can't even access it using curl method https://docs.opnsense.org/development/how-tos/api.html#using-curl

Does your GUI work? See, API not broken...


Cheers,
Franco

You're right, my mistake

[hemin]

I have the same issue but IMO this is some kind of API issue in 21.1.* . I can't even access it using curl method https://docs.opnsense.org/development/how-tos/api.html#using-curl

Does your GUI work? See, API not broken...


Cheers,
Franco

You're right, my mistake

Hi QBANIN, I have the same problem, API Working but I get home assistant error. Have you solve it?

[Grossartig]

There's some useful information in this thread (scroll to bottom):

https://github.com/home-assistant/core/issues/46346