OpenVPN not work after update OPNSense from 23.1.1_2->23.1.3

[malkovich78]

repeating the same message without offering any information I asked for earlier (https://forum.opnsense.org/index.php?topic=32939.msg159704#msg159704) likely isn't going to lead to an improvement. It was sheer luck silverspy18 mentioned static-challenge, otherwise nothing would have changed until now.

Best regards,

Ad

Tough but fair, I use certifficate-based auth so the following command gived no output:

root@opnsense# grep -r auth-user-pass-verify /var/etc/openvpn/*.conf
root@opnsense#

I attach my edited config and the error logs I get on both sides just in case it helps:

# cat /var/etc/openvpn/server2.conf
dev ovpns2
verb 1
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_server2.pid
script-security 3
daemon openvpn_server2
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
data-ciphers-fallback AES-256-CBC
auth SHA256
up /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup
down /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkdown
local #edited_ip addr#
client-connect "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '2'"
tls-server
server #network mask#
client-config-dir /var/etc/openvpn-csc/2
ifconfig #edited_ip1 ip2#
tls-verify "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '2'"
lport #edited_port#
management /var/etc/openvpn/server2.sock unix
max-clients 2
push "route #edited_route_1 mask#"
push "route #edited_route_2 mask#"
push "route #edited_route_3 mask#"
push "route #edited_route_4 mask#"
push "route #edited_route_5 mask#"
push "route #edited_route_6 mask#"
ca /var/etc/openvpn/server2.ca
cert /var/etc/openvpn/server2.cert
key /var/etc/openvpn/server2.key
dh /usr/local/etc/inc/plugins.inc.d/openvpn/dh.rfc7919
tls-crypt /var/etc/openvpn/server2.tls-crypt
push "dhcp-option DNS #edited_dns_ip#"
push "dhcp-option DOMAIN #edited_localdomain#"
auth-nocache

Server logs:
2023-03-16T16:51:56   Warning   openvpn_server2   #edited IP#:5033 WARNING: 'keysize' is used inconsistently, local='keysize 256', remote='keysize 128'   
2023-03-16T16:51:56   Warning   openvpn_server2   #edited IP#:5033 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1569', remote='link-mtu 1553'

Client logs:
2023-03-16 16:54:50 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1553', remote='link-mtu 1569'
2023-03-16 16:54:50 WARNING: 'keysize' is used inconsistently, local='keysize 128', remote='keysize 256'
2023-03-16 16:54:50 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 2048 bit RSA, signature: RSA-SHA256
2023-03-16 16:54:50 [#edited IP#] Peer Connection Initiated with [AF_INET]#edited IP#:#edited_port#
2023-03-16 16:54:51 MANAGEMENT: >STATE:1678982091,GET_CONFIG,,,,,,
2023-03-16 16:54:51 SENT CONTROL [#edited IP#]: 'PUSH_REQUEST' (status=1)
2023-03-16 16:54:51 AUTH: Received control message: AUTH_FAILED,Data channel cipher negotiation failed (no shared cipher)
2023-03-16 16:54:51 TCP/UDP: Closing socket

Error logs say clear that there was a cipher negotiation so I changed to AES-256-GCM instead of CBC and now it works, seems that AES-256-CBC is considered deprecated now; previous log messages (Interrupted system call (code=4)) confused me...

Thanks for your help.

[malkovich78]

Well, full reports would help like Ad suggested. Your issue is probably https://github.com/opnsense/core/commit/4b2b60050

Not sure what OpenVPN is expecting here but we will be reverting to the original (deprecated) behaviour and hope they keep supporting it onwards. ;)


Cheers,
Franco

Bingo! that explains why it wasn't taking notice of following line in my config:

data-ciphers-fallback AES-256-CBC

Thanks and regards

[yourfriendarmando]

Hi All

I confirm this broke upgrading from CE 23.1.3 to 23.1.3_4 . They are for my parents house and small non profit. They haven't complained about no internet as they don't use the VPN, but I do for remote checkups.

For 8 clients running on BE, and myself (9), this has not affected any of them.

Is there an easy way to revert? Or will a fix automate rollback to a working version?

[silverspy18]

@ AdSchellevis, I've confirmed that the patch b5289522 fixed my problem with the client option 'static-challenge' failing. Thanks!

[yourfriendarmando]

In my case, I was not able to check because I forgot my laptop, but someone trying to connect, caused a log entry of:

Certificate max depth of 1 exceeded. I always set this value from One (Client + Server) to Two (Client + Intermediate + Server). This allowed the client connection to be successfully established.

I followed instruction in the Manuel about setting up clients, and made a CA, Intermediate CA, and self signed cert. Perhaps the setting of 1 was not actually enforced previously? By setting it to 2, it worked, as I guess that was my own presenting setup of both server and client?

It works, hopefully this gets my issue out of the way in favor of, what I'm sure will be an overhaul, of the interface for OpenVPN 2.6.x series.

Thanks all

[kpurrucker]

The Patch works also on our production system with 100 users. So I can confirm, that the patch solve the problem on systems with Auth via Password + TOTP + Client Cert. Thanks to @AdSchellevis!

[guest31444]

Hello Guys,

is that issue a firmware bug or a missconfig?

when will this problem be solved by an update?

thank you

[benyamin]

@mara, the patch was committed to master three weeks ago.

You might want to update to the latest revision (23.1.5_4) announced here.

You will likely want to consider the patches here too.

[eugenmayer]

For me, upgrading from 23.1.6 to 23.1.7 broke the OpenVPN authentication again. I have seen that there is a legacy feature maintained for cipher in the patch.

From the logs i would say it is the same / very similar issue.

[eugenmayer]

AFAICS this is related to the upgrade to OpenVPN 2.6.3, which is included in 23.1.7 (compared to 2.5.8 in 23.1.6) - the server does crash for us (the entire daemon) when linux clients connect (about 100). 2.5.8 does work just fine combined with 23.1.6

One has to downgrade opnsense to 23.1.6, since it seems like 23.1.7 changed the config for OpenVPN (so it is compatible with 2.6.3?)

[mimugmail]

Without posting any logs its impossible to help :)

[benyamin]

I have seen that there is a legacy feature maintained for cipher in the patch.

AFAICS this is related to the upgrade to OpenVPN 2.6.3, which is included in 23.1.7 (compared to 2.5.8 in 23.1.6) - the server does crash for us (the entire daemon) when linux clients connect (about 100). 2.5.8 does work just fine combined with 23.1.6

One has to downgrade opnsense to 23.1.6, since it seems like 23.1.7 changed the config for OpenVPN (so it is compatible with 2.6.3?)

It's likely because the compatibility behaviour for --cipher present in v2.5 may have been dropped in v2.6. Previously, the algorithm at --cipher was appended to the list at --data-ciphers. Also, the --data-ciphers-fallback option is really only meant to be applicable to v2.3 peers using --enable-small.

You may want to use both the --cipher and the --data-ciphers-fallback options. Depending on compatibility modes it should pick up one of them.

If that doesn't work you can use --data-ciphers AES-256-GCM:AES-128-GCM:?CHACHA20-POLY1305:<the cipher you need> which will retain AEAD options should your peer upgrade from the legacy cipher.

For more info, there's this topic and this issue.

[franco]

You may want to use both the --cipher and the --data-ciphers-fallback options. Depending on compatibility modes it should pick up one of them.

Ad and me suspected that this might work, but seems to be far from a desired outcome and likely prone to subtle issues depending on how it's being implemented now or in the future.


Cheers,
Franco

[eugenmayer]

data-ciphers-fallback does work, we tried that. And this was the issue here.
The reason 23.1.3 failed initially was, that cipher was replaced using data-ciphers (only), which will not work for 2.3 OpenVPN clients if those exists (only from 2.4+). And additional issue is, that the default allowed cyphers changed with the server 2.5, blocking AES-128-CBC - and this was the other issue why people with 2.4 clients could potentially not connect with the old 23.1.3 (pre-patched;v.

AFAICS even with 23.1.7 ciphers is still used in the server config - we removed that by using 'none' and using data-ciphers instead in the custom section, with a list of cyphers are clients need (and thus a road to upgrade ciphers) - this allows all our clients to connect and would be the proper fix for the variant introduced 21.1.3 (since as stated, ciphers itself is deprecated and will be removed with 2.7 AFAIR)

The other issue i meanioned, that the VPN server is crashing under 2.6.3 is something new and not related - i just was not aware while I was investigating. It is a new issue and related to 2.6.x upgrade with 23.1.7.

I created https://forum.opnsense.org/index.php?topic=34052.0 to separate the issues.

[benyamin]

You may want to use both the --cipher and the --data-ciphers-fallback options. Depending on compatibility modes it should pick up one of them.

Ad and me suspected that this might work, but seems to be far from a desired outcome and likely prone to subtle issues depending on how it's being implemented now or in the future.

From OpenVPN v2.6.0, --cipher is always ignored in TLS mode and only used for PSK mode and when using --compat-mode version, neither of which are recommended. Clients connecting to remote legacy servers might still need it depending on the server version.

AFAICS even with 23.1.7 ciphers is still used in the server config - we removed that by using 'none' and using data-ciphers instead in the custom section, with a list of cyphers are clients need (and thus a road to upgrade ciphers) - this allows all our clients to connect and would be the proper fix for the variant introduced 21.1.3 (since as stated, ciphers itself is deprecated and will be removed with 2.7 AFAIR)

Presuming you're not doing so already, consider prefixing the --data-ciphers cipher-list with AES-256-GCM:AES-128-GCM:?CHACHA20-POLY1305:. That way as clients progress through your upgrade path they will also be able to make use of an AEAD cipher. Sacrifices of the list's constituent ciphers might be necessary to ensure it remains less than 128 characters in length...