openvpn not working after upgrade

[wessjan]

Hi there,
I've just upgraded my installation from 22.1.10 to 22.7.2. Everything worked fine so far, but openvpn clients couldn't connect. Every time the openvpn is restarted the logs show the same warnings, which for me looks like the process can't read the files (server1.*) in the config folder (/var/etc/openvpn). The process is running as root, the access for the folder is 750 root/wheel and for the files 600 root/wheel. In the web GUI the config can be read, but after saving (which implies a restart for openvpn)  the same warnings are shown in the log. Client connects are refused as the CRL file is not readable.

content of folder:

Code: [Select]

root@mfw005:/var/etc/openvpn # ll
total 40
-rw-------  1 root  wheel  1757 Aug 20 00:52 server1.ca
-rw-------  1 root  wheel  1862 Aug 20 00:52 server1.cert
-rw-------  1 root  wheel  1352 Aug 20 00:52 server1.conf
-rw-------  1 root  wheel  1432 Aug 20 00:52 server1.crl-verify
-rw-------  1 root  wheel  1704 Aug 20 00:52 server1.key
srwxrwxrwx  1 root  wheel     0 Aug 20 00:52 server1.sock=
warnings in log on service start:

Code: [Select]

2022-08-20T00:52:17 Warning openvpn Could not determine IPv4/IPv6 protocol. Using AF_INET6
2022-08-20T00:52:16 Warning openvpn CRL: cannot read CRL from file /var/etc/openvpn/server1.crl-verify
2022-08-20T00:52:16 Warning openvpn OpenSSL: error:0909006C:PEM routines:get_name:no start line
2022-08-20T00:52:16 Warning openvpn NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2022-08-20T00:52:16 Warning openvpn NOTE: your local LAN uses the extremely common subnet address 192.168.0.x or 192.168.1.x. Be aware that this might create routing conflicts if you connect to the VPN server from public locations such as internet cafes that use the same subnet.
2022-08-20T00:52:15 Warning openvpn DEPRECATED OPTION: --cipher set to 'AES-256-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM). Future OpenVPN version will ignore --cipher for cipher negotiations. Add 'AES-256-CBC' to --data-ciphers or change --cipher 'AES-256-CBC' to --data-ciphers-fallback 'AES-256-CBC' to silence this warning.
2022-08-20T00:52:15 Warning openvpn WARNING: --topology net30 support for server configs with IPv4 pools will be removed in a future release. Please migrate to --topology subnet as soon as possible.
2022-08-20T00:52:15 Warning openvpn WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
errors in log on client connect:

Code: [Select]

2022-08-20T00:47:28 Error openvpn 192.168.***.***:43982 Fatal TLS error (check_tls_errors_co), restarting
2022-08-20T00:47:28 Error openvpn 192.168.***.***:43982 TLS Error: TLS handshake failed
2022-08-20T00:47:28 Error openvpn 192.168.***.***:43982 TLS Error: TLS object -> incoming plaintext read error
2022-08-20T00:47:28 Error openvpn 192.168.***.***:43982 TLS_ERROR: BIO read tls_read_plaintext error
2022-08-20T00:47:28 Error openvpn 192.168.***.***:43982 OpenSSL: error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed
2022-08-20T00:47:28 Error openvpn 192.168.***.***:43982 VERIFY ERROR: CRL not loaded
2022-08-20T00:47:21 Error openvpn fdf0:***:e554 Fatal TLS error (check_tls_errors_co), restarting
2022-08-20T00:47:21 Error openvpn fdf0:***:e554 TLS Error: TLS handshake failed
2022-08-20T00:47:21 Error openvpn fdf0:***:e554 TLS Error: TLS object -> incoming plaintext read error
2022-08-20T00:47:21 Error openvpn fdf0:***:e554 TLS_ERROR: BIO read tls_read_plaintext error
2022-08-20T00:47:21 Error openvpn fdf0:***:e554 OpenSSL: error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed
2022-08-20T00:47:21 Error openvpn fdf0:***:e554 VERIFY ERROR: CRL not loaded

Any idea to get that fixed?

best regards Jan

[Fright]

Hi!
same issue?
https://forum.opnsense.org/index.php?topic=29567.0
some debug info and errors protection added with

Code: [Select]

opnsense-patch 3c53058but the main reason (lack of non-RSA key type algos support in phpseclib2) has not yet been resolved.
it is necessary to create a certification authority with a RSA key or create CRLs outside of OPN and import them

[wessjan]

Hi Fright,
thanks for the link, I missed that post yesterday, when searching for "openvpn" issues.

Indeed removing the CRL from the server config solved the issue for the clients, they can connect again. But the other warnings still appear.

Some more details from my system: the CA is hosted on another box, the key was created with RSA. On the OPNsense box I use a intermediate CA signed by this CA, created with RSA as well.

The CRL is created on the other system automatically using the CA key (not the intermediate CA) and pushed wherever it's needed. For my OPNsense sytem it's copied to /var/etc/openvpn/server1.crl-verify by cron using scp.

If I can help locating the error with more information, please let me know.
 
CA Certificate:

Code: [Select]

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = DE, ...
        Validity
            Not Before: Jan 26 22:43:06 2015 GMT
            Not After : Jan 23 22:43:06 2025 GMT
        Subject: C = DE, ...
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    ...
                Exponent: ...
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                ...
            X509v3 Authority Key Identifier:
                keyid: ...
                DirName:/C=DE ...
                serial: ...

            X509v3 Basic Constraints:
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
       ...
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Intermidiate CA Cert:

Code: [Select]

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 57 (0x39)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = DE, ...
        Validity
            Not Before: Mar 11 22:27:48 2022 GMT
            Not After : Mar  8 22:27:48 2032 GMT
        Subject: C = DE, ... CN = CA-mfw005, ...
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    ...
                Exponent: ...
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                ...
            X509v3 Authority Key Identifier:
                keyid: ...
                DirName:/C=DE ...
                serial: ...

            X509v3 Basic Constraints:
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
     ...
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
CRL:

Code: [Select]

Certificate Revocation List (CRL):
        Version 1 (0x0)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = DE,  ...
        Last Update: Aug 14 04:47:01 2022 GMT
        Next Update: Sep 13 04:47:01 2022 GMT
Revoked Certificates:
    Serial Number: 04
        Revocation Date: Oct 12 16:16:58 2017 GMT
    Serial Number: 06
        Revocation Date: Dec 23 11:40:31 2017 GMT
    Serial Number: 08
        Revocation Date: Dec  5 21:43:35 2018 GMT
    Serial Number: 09
        Revocation Date: Oct 21 21:37:37 2020 GMT
    Serial Number: 0F
        Revocation Date: Oct 21 21:38:18 2020 GMT
    Serial Number: 10
        Revocation Date: Oct 21 21:38:32 2020 GMT
    Serial Number: 11
        Revocation Date: Oct 21 21:38:59 2020 GMT
    Serial Number: 13
        Revocation Date: Oct 21 21:39:17 2020 GMT
    Serial Number: 21
        Revocation Date: Mar 13 23:20:18 2018 GMT
    Serial Number: 27
        Revocation Date: Oct 21 21:42:07 2020 GMT
    Serial Number: 2D
        Revocation Date: Jan  1 17:45:14 2019 GMT
    Serial Number: 38
        Revocation Date: Feb 21 15:29:12 2022 GMT
    Signature Algorithm: sha256WithRSAEncryption
    ...

br Jan

[Fright]

The CRL is created on the other system automatically using the CA key (not the intermediate CA) and pushed wherever it's needed.

then this is definitely not the same issue

...and pushed wherever it's needed

but the timestamp "Aug 20 00:52 server1.crl-verify" shows that this file was generated along with the others during ovpn instance startup, not placed separately?
so I would still pay attention to the state of the CRL, which is specified in the server settings.
this CRL still should generate valid (may be "empty") CRL (not empty file)

[wessjan]

You are right. OPNsense changes the crl file upon restart.

the content of the original file copied to the box to server1. crl-verify is:

Code: [Select]

-----BEGIN X509 CRL-----
MIIC4jCCAcowDQYJKoZIhvcNAQELBQAwgacxCzAJBgNVBAYTAkRFMQswCQYDVQQI
...
sZ65cxZXWgrR1nHzXCta2wkxZOP1Lg==
-----END X509 CRL-----

After restarting the openvpn service the content in server1. crl-verify has changed to:

Code: [Select]

TUlJQzRqQ0NBY293RFFZSktvWklodmNOQVFFTEJRQXdnYWN4Q3pBSkJnTlZCQVlUQWtSRk1Rc3dDUVlEVlFRSQ0KRXd
...
XZ3JSMW5IelhDdGEyd2t4Wk9QMUxnPT0NCi0tLS0tRU5EIFg1MDkgQ1JMLS0tLS0NCg==

If I checked that with openssl and got an error:

Code: [Select]

root@mfw005:/var/etc/openvpn # openssl crl -inform PEM -text -noout -in server1.crl-verify
unable to load CRL
34389172224:error:0909006C:PEM routines:get_name:no start line:/usr/src/crypto/openssl/crypto/pem/pem_lib.c:745:Expecting: X509 CRL

When using the GUI to set an imported CRL and paste the X.509 crl in the text box and save, the server1.crl-verify contains the correct X.509 CRL. When I edit it in the GUI, the content of the text box changed to:

Code: [Select]

LS0tLS1CRUdJTiBYNTA5IENSTC0tLS0tDQpNSUlDNGpDQ0Fjb3dEUVlKS29aSWh2Y05BUUVMQlFBd2dhY3hDekFKQm
....
jY1Y3haWFdnclIxbkh6WEN0YTJ3a3haT1AxTGc9PQ0KLS0tLS1FTkQgWDUwOSBDUkwtLS0tLQ0K
and the content of the server1.crl-verify has changed accordingly. Using openssl to check this figures out again, that this is not a valid CRL file.

I've created a internal test CA, server certificate, client certificate and crl on the OPNsense box and setup another openvpn server with these credentials. Of course the crl is empty, but the content of the server2.crl-verify is always a valid X.509 CRL.

With the change from 22.1 to 22.7 something has changed with imported X.509 CRLs. This should be fixed, as using a crl is an important security feature for operating a VPN and if you are using other systems with the same CA it's not possible to switch to an internal implementation just on the OPNsense box.

I'm not familiar in raising a bug ticket for OPNsense, but when getting some instruction how to, I'll do.

cheers Jan
 

[Fright]

ah, in this case, it seems that the problem is in trying to edit the imported crl and hitting save button with broken "CRL data" field (data from config not BASE64 decoded) (if you go into editing mode for the imported list, you must insert a valid pem crl data  before saving).
I'll try to see if I can make a pr for this issue.

if you did not try to edit the imported crl is everything worked as expected?
(I checked on a similar scheme: external root and subordinate certificates and an imported CRLL: the correct file for vpn is generated)

[wessjan]

To avoid any side effects I started from scratch. First removed the imported CRL under Trusts/Revocation, then created a new one there as imported from my root CA and pasted the X.509 CRL into the text box.

The openvpn server was at that moment running with no crl checking configured. After editing the servers config and enabled crl checking with the new CRL the openvpn server restarted without the warnings regarding the CRL and also the clients can still connect. Also after a manual restart of the service the crl file stays as expected.

thanks for your help!

Jan

[Fright]

Good to know, thanks!
For the reference: ticket for the mentioned issue of an existing imported crl display format (and possible crl breakage on save):
https://github.com/opnsense/core/pull/5965