Just upgraded to 24.10 BE on my test device. Upgrade itself went fine, but when I go to check for updates, I'm getting a CRL error:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10 at Thu Oct 17 10:50:46 EDT 2024
Fetching subscription information, please wait... Could not load CRL file /tmp/libfetch_crl.24101710
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/subscription: Authentication error
Fetching changelog information, please wait... Could not load CRL file /tmp/libfetch_crl.24101710
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/sets/changelog.txz: Authentication error
Updating OPNsense repository catalogue...
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/meta.txz: Authentication error
repository OPNsense has no meta file, using default settings
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.pkg: Authentication error
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
Could not load CRL file /tmp/libfetch_crl.24101710
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.txz: Authentication error
Unable to update repository OPNsense
Error updating repositories!
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
This is what's in /tmp/libfetch_crl.24101710:
# [i] fetch certificate for https://opnsense-update.deciso.com
Figured a new file would be updated on the hour and it was. libfetch_crl.24101711 also only contains that single line.
Thanks. -Steve
I received an error before it rebooted It rebooted rather quicker than expected
But says it's updated and shows the posted above
Can you fetch http://x1.c.lencr.org/ from the box? It looks like you cannot.
Cheers,
Franco
It looks like I can, but strangely it doesn't contain any revoked certs:
root@testrange:~ # curl http://x1.c.lencr.org/ --output /tmp/test.crl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 717 100 717 0 0 9425 0 --:--:-- --:--:-- --:--:-- 9434
root@testrange:~ # openssl crl -in /tmp/test.crl -inform DER -text -noout
Certificate Revocation List (CRL):
Version 2 (0x1)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Internet Security Research Group, CN = ISRG Root X1
Last Update: Feb 5 00:00:00 2024 GMT
Next Update: Jan 4 23:59:59 2025 GMT
CRL extensions:
X509v3 Authority Key Identifier:
79:B4:59:E6:7B:B6:E5:E4:01:73:80:08:88:C8:1A:58:F6:E9:9B:6E
X509v3 CRL Number:
104
No Revoked Certificates.
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
59:26:d6:a5:01:52:f5:e1:20:f8:e7:5d:6d:28:5c:a1:6f:39:
1e:ee:92:a8:4d:07:f9:a4:65:af:37:db:f8:a9:4f:df:a1:b4:
96:e5:61:1a:84:3c:03:66:0d:4f:6c:33:1c:97:b1:e5:33:9e:
4a:d9:1e:88:c7:42:8e:fd:36:21:24:e6:a0:87:b0:d2:c4:34:
41:7c:d3:68:9c:50:f2:a5:a6:09:8c:8b:c2:62:63:dc:26:a4:
12:ae:c3:81:65:c0:44:2a:35:01:49:b2:cd:59:6a:e7:5d:f1:
1f:63:84:aa:a1:53:3a:5f:7f:f3:9a:ed:42:4b:64:21:52:fa:
9d:e9:b9:af:bb:c7:5c:e4:78:3c:47:f3:be:16:78:c4:23:63:
c1:6a:e9:8e:65:31:9b:00:24:0f:91:20:98:1f:47:55:ca:ab:
6a:72:ad:ac:b9:c0:f9:3c:4f:1b:46:58:d8:50:8f:e7:13:7b:
ff:fb:5f:8b:c1:ba:01:97:37:77:34:20:a8:d5:4d:b0:9c:f2:
8f:6d:22:b2:dd:5f:05:b6:2c:de:99:a2:b6:ea:ef:59:64:d5:
c1:b0:7f:80:45:cc:68:87:7c:63:eb:63:07:f1:49:1a:8f:38:
e4:05:2d:7c:e0:42:98:ae:07:07:8b:f7:9c:3b:a9:09:70:bf:
8f:52:d3:30:ea:df:42:67:88:6b:d2:de:ab:3d:28:a4:7a:d7:
7d:bb:82:6f:6a:10:96:01:4a:3f:81:d7:e1:e3:5a:91:58:9e:
2d:f2:f9:5e:58:cf:ce:63:a3:bd:46:8f:0c:97:6c:4f:97:d5:
48:de:9c:cb:57:c3:9a:c6:a2:92:78:e6:05:3d:d5:4e:14:d9:
f8:f4:09:9e:d2:fe:13:38:5b:e9:af:0a:ec:92:e7:bf:ee:5a:
33:48:ee:31:82:d7:6f:0b:cd:ec:aa:db:66:9f:d8:a1:63:20:
57:7b:76:aa:d0:d6:a5:1e:c9:44:45:dd:3c:18:bd:6f:05:b8:
19:58:a0:e9:c5:8a:58:70:3b:e4:22:bf:0d:c8:a3:e0:53:a6:
7f:2b:a6:39:14:ad:2b:0d:b7:4a:46:d2:78:21:67:6b:25:33:
23:d6:ab:17:80:bb:66:22:ec:ee:6d:b1:e5:01:ae:4e:5b:5c:
3b:35:54:3e:5a:94:51:f5:81:eb:cb:10:ca:d6:39:7e:17:ae:
f0:4d:25:81:64:cd:b6:06:09:ea:75:eb:0e:06:e5:a4:c0:1e:
0e:24:9f:33:bf:fd:1f:12:48:57:60:e1:a4:e8:aa:b2:30:e9:
ec:e0:52:76:44:4e:bd:42:69:69:b5:de:51:ef:84:a4:16:19:
49:a2:2b:d2:3d:62:b4:6e
root@testrange:~ #
Empty CRL is ok but it has to be there. Can you run this manually?
# /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
Just tested again from here and I get it loaded into the CRL file so that it proceeds to talk to the update server.
We will likely switch the chain away from LE, but that won't address the issue of not being able to download CRLs. This is a certification requirement so we can't just disable it when it fails. But the OpenSSL situation itself is sticky. My favourite mail exchange on the subject:
https://openssl-users.openssl.narkive.com/qKrxQx5U/certificate-crls-x509-v-err-unable-to-get-crl
Thanks,
Franco
root@testrange:~ # /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
# [i] fetch certificate for https://opnsense-update.deciso.com
[!!] Chain fetch failed for https://opnsense-update.deciso.com
I had checked the box for Auto fetch CRLs in System:Trust:Settings, and the 1pm update of /tmp/libfetch_crl.24101713 has CRLs in the file now:
root@testrange:/tmp # cat libfetch_crl.24101713
-----BEGIN X509 CRL-----
MIIBVDCB3AIBATAKBggqhkjOPQQDAzBIMQswCQYDVQQGEwJERTEVMBMGA1UEChMM
RC1UcnVzdCBHbWJIMSIwIAYDVQQDExlELVRSVVNUIEVWIFJvb3QgQ0EgMSAyMDIw
Fw0yNDAxMTUxNDAwNDlaFw0yNTAxMTYxMzU5NDlaMDEwLwIQaOXtSmaheM68bssu
...
sptVq1l7Np3JGm7LV6UkiPfEHWbtz3BXrGWe8TYJacs1ekmk06yHivnEFZHUVNAf
CzjrFOBpb2V0giVc4FT1pFyyAsHYJ8CEACufMddpJcnmqsm+u5v5s5ECvwcIVcrV
iyfpHYL9lVi4VI7b2k++Adl0fJXojykktFeycNFYCYP/fw==
-----END X509 CRL-----
# [i] fetch certificate for https://opnsense-update.deciso.com
Check for Updates appears to be working now:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10 at Thu Oct 17 13:16:14 EDT 2024
Fetching subscription information, please wait... No CRL was provided for /CN=opnsense-update.deciso.com
No CRL was provided for /C=US/O=Let's Encrypt/CN=R11
No CRL was provided for /C=US/O=Internet Security Research Group/CN=ISRG Root X1
done
Fetching changelog information, please wait... No CRL was provided for /CN=opnsense-update.deciso.com
No CRL was provided for /C=US/O=Let's Encrypt/CN=R11
No CRL was provided for /C=US/O=Internet Security Research Group/CN=ISRG Root X1
No CRL was provided for /CN=opnsense-update.deciso.com
No CRL was provided for /C=US/O=Let's Encrypt/CN=R11
No CRL was provided for /C=US/O=Internet Security Research Group/CN=ISRG Root X1
done
Updating OPNsense repository catalogue...
No CRL was provided for /CN=opnsense-update.deciso.com
No CRL was provided for /C=US/O=Let's Encrypt/CN=R11
No CRL was provided for /C=US/O=Internet Security Research Group/CN=ISRG Root X1
Fetching meta.conf: . done
No CRL was provided for /CN=opnsense-update.deciso.com
No CRL was provided for /C=US/O=Let's Encrypt/CN=R11
No CRL was provided for /C=US/O=Internet Security Research Group/CN=ISRG Root X1
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 856 packages processed.
All repositories are up to date.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Checking for upgrades (0 candidates): . done
Processing candidates (0 candidates): . done
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
Still seeing Chain fetch failed from the python script, but it's now printing the CRLs as well:
root@testrange:/tmp # /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
# [i] fetch certificate for https://opnsense-update.deciso.com
[!!] Chain fetch failed for https://opnsense-update.deciso.com
-----BEGIN X509 CRL-----
MIIBVDCB3AIBATAKBggqhkjOPQQDAzBIMQswCQYDVQQGEwJERTEVMBMGA1UEChMM
RC1UcnVzdCBHbWJIMSIwIAYDVQQDExlELVRSVVNUIEVWIFJvb3QgQ0EgMSAyMDIw
Fw0yNDAxMTUxNDAwNDlaFw0yNTAxMTYxMzU5NDlaMDEwLwIQaOXtSmaheM68bssu
...
This is a "workaround" of sorts. The problem is OpenSSL fails hard if the CRL file is empty, but it gladly accepts multiple ones even if unrelated. It does insist on one, even though that's technically wrong as it would later complain about the missing CRL anyway (the latter is ok, the former fail is more dangerous when you don't know where the user is going to go). I don't think any of this OpenSSL-specific behaviour can be made much better than it is (or nobody will be willing to accept it for "security" reasons).
For now we need to know what the issue during your fetch is:
# opnsense-patch https://github.com/opnsense/core/commit/372c9c98
# /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
Thanks,
Franco
root@testrange:~ # /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
# [i] fetch certificate for https://opnsense-update.deciso.com
[!!] Chain fetch failed for https://opnsense-update.deciso.com (HTTPSConnectionPool(host='opnsense-update.deciso.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x2cedeac42d10>, 'Connection to opnsense-update.deciso.com timed out. (connect timeout=0.1)')))
-----BEGIN X509 CRL-----
MIIBVDCB3AIBATAKBggqhkjOPQQDAzBIMQswCQYDVQQGEwJERTEVMBMGA1UEChMM
RC1UcnVzdCBHbWJIMSIwIAYDVQQDExlELVRSVVNUIEVWIFJvb3QgQ0EgMSAyMDIw
...
Ok, timeout=0.1 looks like an obvious problem. Another patch coming up in a minute. Thanks!
# opnsense-patch https://github.com/opnsense/core/commit/70df0a15f7e
# /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
Cheers,
Franco
root@testrange:/tmp # /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
# [i] fetch certificate for https://opnsense-update.deciso.com
# [i] fetch CRL from http://x1.c.lencr.org/
-----BEGIN X509 CRL-----
MIIBVDCB3AIBATAKBggqhkjOPQQDAzBIMQswCQYDVQQGEwJERTEVMBMGA1UEChMM
RC1UcnVzdCBHbWJIMSIwIAYDVQQDExlELVRSVVNUIEVWIFJvb3QgQ0EgMSAyMDIw
...
I see this additional CRL added to libfetch_crl.24101714:
Certificate Revocation List (CRL):
Version 2 (0x1)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Internet Security Research Group, CN = ISRG Root X1
Last Update: Feb 5 00:00:00 2024 GMT
Next Update: Jan 4 23:59:59 2025 GMT
CRL extensions:
X509v3 Authority Key Identifier:
79:B4:59:E6:7B:B6:E5:E4:01:73:80:08:88:C8:1A:58:F6:E9:9B:6E
X509v3 CRL Number:
104
No Revoked Certificates.
Check for updates is now only showing one No CRL:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10 at Thu Oct 17 14:15:45 EDT 2024
Fetching subscription information, please wait... No CRL was provided for /CN=opnsense-update.deciso.com
done
Fetching changelog information, please wait... No CRL was provided for /CN=opnsense-update.deciso.com
No CRL was provided for /CN=opnsense-update.deciso.com
done
Updating OPNsense repository catalogue...
No CRL was provided for /CN=opnsense-update.deciso.com
Fetching meta.conf: . done
No CRL was provided for /CN=opnsense-update.deciso.com
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 856 packages processed.
All repositories are up to date.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Checking for upgrades (0 candidates): . done
Processing candidates (0 candidates): . done
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
Thanks! -Steve
Hi all
Where are we with this?
I just upgraded both, and this only happens on BE, the most up to date CE appears to be doing just fine.
It's just in my VMs. I'm glad I tried the upgrade, which both succeeded fine. I'll hang back and re-upgrade my 24.4 VM at a later time, try again and make sure it's working fine before I upgrade my main system, as well as of my subscribing clients.
Thank you
@Steve
Thanks for your help. Looks good now on your end. The other CRL related message will go away when we switch to a better chain. LE allegedly thinks their CRL handling is complete but OpenSSL certainly doesn't agree.
@yourfriendarmando
Where we are right now is here:
# opnsense-patch 372c9c98 70df0a15f7e
# /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
Will hotfix tomorrow morning.
Cheers,
Franco
I moved back to community version yesterday
When I woke up this early morning I moved back to BE
all is working now
24.10_1 is available now so it doesn't have to be picked up manually via opnsense-patch anymore.
Cheers,
Franco
I did attempt but from the gui I could not get past the error messages so it could not find anything new
In the past downgrading and upgrading fixed it👍
I'm also at 24.10 BE and receive the could not load CRL file error:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10 at Thu Oct 24 20:15:38 UTC 2024
Fetching subscription information, please wait... Could not load CRL file /tmp/libfetch_crl.24102420
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/subscription: Authentication error
Fetching changelog information, please wait... Could not load CRL file /tmp/libfetch_crl.24102420
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/sets/changelog.txz: Authentication error
Updating OPNsense repository catalogue...
Waiting for another process to update repository OPNsense
Updating SunnyValley repository catalogue...
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
pkg: https://updates.zenarmor.com/opnsense/FreeBSD:14:amd64/24.7/${SUBSCRIPTION}/meta.txz: Authentication error
repository SunnyValley has no meta file, using default settings
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
pkg: https://updates.zenarmor.com/opnsense/FreeBSD:14:amd64/24.7/${SUBSCRIPTION}/packagesite.pkg: Authentication error
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
Could not load CRL file /tmp/libfetch_crl.24102420
pkg: https://updates.zenarmor.com/opnsense/FreeBSD:14:amd64/24.7/${SUBSCRIPTION}/packagesite.txz: Authentication error
Unable to update repository SunnyValley
Error updating repositories!
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
I did attempt
# opnsense-patch 372c9c98 70df0a15f7e
# /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
and the update-crl-fetch.py worked. However, a check for udpates in the GUI still fails.
Just make sure to
# rm /tmp/libfetch_crl.*
And check from the GUI again. If it complains about a file post the contents of it here, e.g.
# cat /tmp/libfetch_crl.24102420
(I'm assuming the file is empty which is a risk with OpenSSL and file-based CRL provider). We're switching to the trust store integration in 24.10.1, but it needs a bit more tweaking at the moment.
Worst case you cannot fetch the CRL because a strict block rule was implemented.
Cheers,
Franco
Thanks! Removing all the various certificates downloaded since the upgrade to 24.10 and re-running an update worked.
# rm /tmp/li
libfetch_crl.24101713 libfetch_crl.24102022 libfetch_crl.24102420
libfetch_crl.24101722 libfetch_crl.24102122 libfetch_crl.24102422
libfetch_crl.24101822 libfetch_crl.24102222 lighttpdcompress/
libfetch_crl.24101922 libfetch_crl.24102322
root@OPNsense:~ # rm /tmp/libfetch_crl.2410*
The tab complete shows roughly one a day since the upgrade on 17 Oct.
I am now at 24.10_7.
Ok, good. I'm hopeful the issue will not be back with the hotfixes in place.
Cheers,
Franco
I'm still having the same problem with 24.10_7:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10_7 at Thu Oct 31 11:42:39 CET 2024
Fetching subscription information, please wait... done
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 856 packages processed.
Updating SunnyValley repository catalogue...
No CRL was provided for /CN=zenarmor.com
No CRL was provided for /C=US/O=Google Trust Services/CN=WE1
No CRL was provided for /C=US/O=Google Trust Services LLC/CN=GTS Root R4
No CRL was provided for /CN=zenarmor.com
No CRL was provided for /C=US/O=Google Trust Services/CN=WE1
No CRL was provided for /C=US/O=Google Trust Services LLC/CN=GTS Root R4
Fetching meta.conf: . done
No CRL was provided for /CN=zenarmor.com
No CRL was provided for /C=US/O=Google Trust Services/CN=WE1
No CRL was provided for /C=US/O=Google Trust Services LLC/CN=GTS Root R4
No CRL was provided for /CN=zenarmor.com
No CRL was provided for /C=US/O=Google Trust Services/CN=WE1
No CRL was provided for /C=US/O=Google Trust Services LLC/CN=GTS Root R4
Fetching packagesite.pkg: ... done
Processing entries: ....... done
SunnyValley repository update completed. 66 packages processed.
All repositories are up to date.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Checking for upgrades (13 candidates): .......... done
Processing candidates (13 candidates): .. done
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
I have deleted all /tmp/libfetch_crl.* files and retried but still get the same error. Current file libfetch_crl.24103111 has valid crl information as it looks like:
# [i] fetch certificate for https://opnsense-update.deciso.com
# [i] fetch CRL from http://cdp.rapidssl.com/RapidSSLTLSECCCAG1.crl
# [i] fetch CRL from http://crl3.digicert.com/DigiCertGlobalRootG3.crl
Is the mentioned hotfix already included in 24.10_7 or will it be available in a later version?
Neither the "same":
These CRL errors are about Zenarmor, not OPNsense repo.
Nor a "problem":
Third party repo CRL verification is not included in 24.10.x yet and you can still connect to Zenarmor just fine.
Cheers,
Franco
Just upgraded some BE to 24.10_7 and am having this same issue, however I suspect there is a different root cause: my 2x BE devices are behind a proxy. They both have proxy environment variables set as per documentation here:
https://docs.opnsense.org/development/backend/configd.html#extending-the-environment
However this does not appear to be passing down the python requests library as it is clearly trying to connect directly.
root@x-y-z:~ # /usr/local/opnsense/scripts/system/update-crl-fetch.py opnsense-update.deciso.com
# [i] fetch certificate for https://opnsense-update.deciso.com
[!!] Chain fetch failed for https://opnsense-update.deciso.com (HTTPSConnectionPool(host='opnsense-update.deciso.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1ee0c0bfa110>: Failed to establish a new connection: [Errno 65] No route to host')))
And from the gui:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10_7 at Thu Nov 7 20:41:56 GMT 2024
Fetching subscription information, please wait... Could not load CRL file /tmp/libfetch_crl.24110720
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/subscription: Authentication error
Fetching changelog information, please wait... Could not load CRL file /tmp/libfetch_crl.24110720
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/sets/changelog.txz: Authentication error
Updating OPNsense repository catalogue...
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/meta.txz: Authentication error
repository OPNsense has no meta file, using default settings
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.pkg: Authentication error
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
Could not load CRL file /tmp/libfetch_crl.24110720
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.txz: Authentication error
Unable to update repository OPNsense
Error updating repositories!
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
Edit: Seems python-requests wants the environment variables in lower case. Added those and now the cmdline script runs without errors. The updates web gui still doesn't, though.
Thanks, looking into it now.
Cheers,
Franco
After editing the script to run fine try to delete the old crl files:
# rm /tmp/libfetch_crl.*
And try the GUI again.
Cheers,
Franco
No, trying again from the GUI just makes it recreate that file again. Each time I rm, the check from the gui brings it back.
However as suggested by another user in this thread, ticking System -> Trust -> Settings -> Autofetch CRLs (which shouldn't have an apostrophe in it :D ) and leaving it for a few hours has done the job. I suspect then from this that the fetch command that is run when you click 'Check for updates' is somehow running in some different environment/context which isn't inheriting the environment variables from configd, where the scheduled check (and pkg, when running 'Check for updates') is.
CRL auto fetch is a workaround for an empty CRL file, but basically you are left without a valid CRL download which we need to debug (and I'm having a deja-vu here writing this).
What's the content of the libfetch_crl file in the error case?
HTTP(S)_PROXY should work in upper and lower case according to the Python code itself.
Cheers,
Franco
Unfortunately in fixing it, I don't have that evidence from when it was broken anymore. I can't roll the environment back as it is semi-live... I'll see if I can recreate it in test.
Going back to the old situation is relatively easy without interfering with operation:
# rm /usr/local/share/certs/ca-crl-upd-* /tmp/libfetch_crl.*
And check for updates again. The new libfetch_crl file should have enough diagnostics to hint at the issue.
Just making sure this is 24.10_7 from updates and not a fresh install 24.10?
Have you set both HTTP_PROXY and HTTPS_PROXY in the configd template? And if you add the lower case ones it starts working or still doesn't?
Cheers,
Franco
Is from updates, not a fresh install. Literally upgraded to 24.10_7 last night and it happened immediately.
I had HTTP_PROXY and HTTPS_PROXY set in the configd template and neither the gui nor update-crl-fetch.py worked. Adding http_proxy and https_proxy in the configd template made update-crl-fetch.py work, but the gui still not. Removing the file from /tmp made no difference - it would just be recreated. I ticked the Auto add CRLs tickbox and went to sleep, and when I woke up it was fixed.
I think that when I looked at the contents of the file, it just contained:
# [i] fetch certificate for https://opnsense-update.deciso.com
Ok probably have to see why the lowercase is better but we did look at the Python code and it said:
https://tedboy.github.io/python_stdlib/_modules/urllib.html#getproxies_environment
I tested and it indeed uses both vars no matter what case the environment var was in (even mixed works). Leads me to believe there may have been a typo in the original ones? Or the environment setup is not correct in configd (misses one?).
Also added https://github.com/opnsense/core/commit/a86c7106ed to verify proxy.conf was added properly and it was (as documented). The only caveat is that configd needs to be restarted when adding/editing/removing the file.
# service configd restart
# configctl configd environment
Cheers,
Franco