OPNsense Forum

Archive => 22.7 Legacy Series => Topic started by: seed on September 01, 2022, 04:07:58 pm

Title: Error after upgrade to 22.7.3
Post by: seed on September 01, 2022, 04:07:58 pm
This is the error i got.
The ACME plugin and OpenVPN Server is not starting anymore.

 PHP Errors:

[01-Sep-2022 16:04:21 Europe/Berlin] PHP Fatal error:  Uncaught phpseclib3\Exception\UnsupportedAlgorithmException: Signature algorithm unsupported in /usr/local/share/phpseclib/File/X509.php:1455
Stack trace:
#0 /usr/local/share/phpseclib/File/X509.php(1412): phpseclib3\File\X509->validateSignatureHelper('rsaEncryption', '-----BEGIN PUBL...', 'id-RSASSA-PSS', '\xA7\xB5[\x84=k\xC2\xD6\x14\x0F(\xA1\xD4r\xE4...', '0\x81\x970\v\x06\t*\x86H\x86\xF7\r\x01\x01...')
#1 /usr/local/share/phpseclib/File/X509.php(1286): phpseclib3\File\X509->validateSignatureCountable(false, 0)
#2 /usr/local/etc/inc/certs.inc(680): phpseclib3\File\X509->validateSignature(false)
#3 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(834): crl_update(Array)
#4 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(1115): openvpn_reconfigure('server', Array)
#5 /usr/local/etc/inc/util.inc(166): openvpn_configure_single('1')
#6 /usr/local/www/status_services.php(43): service_control_start('openvpn', Array)
#7 {main}
  thrown in /usr/local/share/phpseclib/File/X509.php on line 1455
[01-Sep-2022 16:04:29 Europe/Berlin] PHP Fatal error:  Uncaught phpseclib3\Exception\UnsupportedAlgorithmException: Signature algorithm unsupported in /usr/local/share/phpseclib/File/X509.php:1455
Stack trace:
#0 /usr/local/share/phpseclib/File/X509.php(1412): phpseclib3\File\X509->validateSignatureHelper('rsaEncryption', '-----BEGIN PUBL...', 'id-RSASSA-PSS', '\xB2\x1C%\xFD\xA0\x13\x05\xAA\xD3\xF1\x86"\x06v\xA3...', '0\x81\x970\v\x06\t*\x86H\x86\xF7\r\x01\x01...')
#1 /usr/local/share/phpseclib/File/X509.php(1286): phpseclib3\File\X509->validateSignatureCountable(false, 0)
#2 /usr/local/etc/inc/certs.inc(680): phpseclib3\File\X509->validateSignature(false)
#3 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(834): crl_update(Array)
#4 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(1115): openvpn_reconfigure('server', Array)
#5 /usr/local/etc/inc/util.inc(166): openvpn_configure_single('1')
#6 /usr/local/www/status_services.php(43): service_control_start('openvpn', Array)
#7 {main}
  thrown in /usr/local/share/phpseclib/File/X509.php on line 1455
[01-Sep-2022 16:05:17 Europe/Berlin] PHP Fatal error:  Uncaught phpseclib3\Exception\UnsupportedAlgorithmException: Signature algorithm unsupported in /usr/local/share/phpseclib/File/X509.php:1455
Stack trace:
#0 /usr/local/share/phpseclib/File/X509.php(1412): phpseclib3\File\X509->validateSignatureHelper('rsaEncryption', '-----BEGIN PUBL...', 'id-RSASSA-PSS', 'l\xC9\xD3\x9A\xD7\xF0\xFB\xB7\xF5cn`\x17h\xAD...', '0\x81\x970\v\x06\t*\x86H\x86\xF7\r\x01\x01...')
#1 /usr/local/share/phpseclib/File/X509.php(1286): phpseclib3\File\X509->validateSignatureCountable(false, 0)
#2 /usr/local/etc/inc/certs.inc(680): phpseclib3\File\X509->validateSignature(false)
#3 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(834): crl_update(Array)
#4 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(1115): openvpn_reconfigure('server', Array)
#5 /usr/local/etc/inc/util.inc(166): openvpn_configure_single('1')
#6 /usr/local/www/status_services.php(43): service_control_start('openvpn', Array)
#7 {main}
  thrown in /usr/local/share/phpseclib/File/X509.php on line 1455
Title: Re: Error after upgrade to 22.7.3
Post by: fayelund on September 01, 2022, 04:14:49 pm
22.7.3 broke my primary FW, boots, but does not work. So something is really off with it.
But I saw error messages simillar to yours
Title: Re: Error after upgrade to 22.7.3
Post by: seed on September 01, 2022, 04:17:37 pm
i assume this issue is caused by something crl related. once i removed the CRL from my openvpn config openvpn starts again.
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 01, 2022, 04:21:50 pm
EC in but RSA-PSS out? Feels like playing whac-a-mole.

We will look into this tomorrow.

And please don't cross-post "mine is broken" to a detailed report. Thanks. ;)


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 01, 2022, 04:42:29 pm
According to phpseclib this is the relevant bit:

https://github.com/phpseclib/phpseclib/blob/2f0b7af658cbea265cbb4a791d6c29a6613f98ef/phpseclib/File/X509.php#L1434-L1457

This indicates the first element of validateSignatureHelper() should be "id-RSASSA-PSS" but it actually is "rsaEncryption". I'm not sure which is correct, but as far as I know PSS is for signature only so this may be wrong in the library code?


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: athurdent on September 01, 2022, 04:43:18 pm
Failed here, too. Had to revert my VM snapshot
First screenshot shows a sync error right before reboot I never had. Also not when turning it off 1 minute before on the older version to take a snapshot.
Second screenshot shows where it's stuck after the reboot.
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 01, 2022, 04:56:59 pm
Pulled update for now, patch likely going to be https://github.com/opnsense/core/commit/3b39e2d1f6 but this is fishy library handling as the certificate in question seemed to work. It's imported, but no complaints before, especially not pre-phpseclib when we had "native" CLR patching in PHP 7.


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: i81b4u on September 01, 2022, 05:35:40 pm
Pulled update for now, patch likely going to be https://github.com/opnsense/core/commit/3b39e2d1f6 but this is fishy library handling as the certificate in question seemed to work. It's imported, but no complaints before, especially not pre-phpseclib when we had "native" CLR patching in PHP 7.

Did this cause:

Code: [Select]
Checking packages: ........................
opnsense-22.7.3 version mismatch, expected 22.7.2
Checking packages: ..
opnsense-lang-22.7.3 version mismatch, expected 22.7.1
Checking packages: .
opnsense-update-22.7.3 version mismatch, expected 22.7.2
Checking packages: .............
php80-phpseclib-3.0.14 version mismatch, expected 2.0.37
Checking packages: ......................... done
***DONE***

Apart from that I experienced no issues after the upgrade.
Title: Re: Error after upgrade to 22.7.3
Post by: Fright on September 01, 2022, 05:50:28 pm
@franco
how stupid is my idea?:
phpseclib3  shifted the paradigm a bit (https://github.com/phpseclib/phpseclib/issues/1522) and now you have to read the public key, take the signature algorithm from there and explicitly specify it (
Code: [Select]
$ca_cert = $ca_cert->withHash($pub_key_sign_algo);) before signing (otherwise the library will force PSS ("It's doing PSS, as is, because PSS is th default padding scheme that RSA keys utilize") and create a "broken" CRL)
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 01, 2022, 07:41:04 pm
@i81b4u looks normal since you caught the 22.7.3 and it insists on having lower packages when mirror wants to publish 22.7.2 again.

@Fright before working around phpseclib let's first generate a test certificate for this behaviour and then try to fix phpseclib itself. The validation may be off here / trying to enforce something it doesn't have to: the code checks $publicKeyAlgorithm but never checks $signatureAlgorithm in PSS case assuming it's correct. If it just checked $signatureAlgorithm and ignore $publicKeyAlgorithm it might just start working and we can discuss with upstream.


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 01, 2022, 08:26:02 pm
Back online as 22.7.3_2.


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: seed on September 01, 2022, 09:05:05 pm
Thank you for the very quick response.
In the meantime, I had reinstalled my firewall and upgraded to 22.7.2.

I have now tested the update again and could not find any errors on my system.
Title: Re: Error after upgrade to 22.7.3
Post by: Fright on September 02, 2022, 05:21:48 pm
sorry. a little confused.
so is it fixed?
(imho it does not look like this and there are upstream issues  .. or i missed some?)
Title: Re: Error after upgrade to 22.7.3
Post by: Fright on September 03, 2022, 12:13:02 pm
@franco
Hi.
As far as I can understand, the problem has not been solved yet. And I think it's in the phpseclib validateSignatureHelper()  function. Does this make sense in your opinion?
https://github.com/kulikov-a/phpseclib/commit/bf4487c504ea5ea2c36ec7cd0e01fed05b41a6fd

I continue testing, but the CRLs of internal CAs (RSA and EC) works with these.
I will check on various imported CA
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 05, 2022, 10:03:14 am
Patch looks sane to me, better than the other convoluted approach. There is the issue of trying to limit EC stuff to EC keys for example but the key should be loaded regardless and it could return early if the key cannot be loaded?


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: PotatoCarl on September 05, 2022, 10:14:25 am
Hi
I saw here also a booting problem. OPNSense came up, but most services did not work. Proxy only erratic, no VPN, no ACME, no ICAP, no Scruita....
I had to physically power down the FW hardware, start it again and restart services until everything came back up.
HOWEVER, OPENVPN is not working with any client anymore.

I cannot connect.

I get in the protocol these messages:
2022-09-05T10:13:21   Error   openvpn   87.191.224.208:34795 TLS Error: TLS handshake failed   
2022-09-05T10:13:21   Error   openvpn   87.191.224.208:34795 TLS Error: TLS object -> incoming plaintext read error   
2022-09-05T10:13:21   Error   openvpn   87.191.224.208:34795 TLS_ERROR: BIO read tls_read_plaintext error   
2022-09-05T10:13:21   Error   openvpn   87.191.224.208:34795 OpenSSL: error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed   
2022-09-05T10:13:21   Error   openvpn   87.191.224.208:34795 VERIFY ERROR: CRL not loaded   

On the devices "NETWORK_EOF_ERROR" (Android), time out on various linux machines.

Please help! A minor upgrade should not brake the firewall so completely!
Title: Re: Error after upgrade to 22.7.3
Post by: PotatoCarl on September 05, 2022, 10:16:24 am
Okay, after removing the CRL as a matter of fact, the VPN did connect again.

This is a very strange issue, no?
Title: Re: Error after upgrade to 22.7.3
Post by: Fright on September 05, 2022, 12:51:12 pm
@franco
Quote
it could return early if the key cannot be loaded?
yes, the function seems to throw NoKeyLoadedException
Quote
There is the issue of trying to limit EC stuff to EC keys for example
sorry, i'm not sure i understand ..
Title: Re: Error after upgrade to 22.7.3
Post by: franco on September 05, 2022, 09:46:19 pm
Quote
There is the issue of trying to limit EC stuff to EC keys for example
sorry, i'm not sure i understand ..

Switch case was nested so only signatures matching EC would be selectable for it. With your patch that's no longer the case.


Cheers,
Franco
Title: Re: Error after upgrade to 22.7.3
Post by: Fright on September 06, 2022, 10:48:18 am
ah! now i understand, thanks! )