TLS Error: TLS handshake failed

[s.messias]

Hello everyone.

I'm new to this world so please be pattient :P

I have a private server in the cloud in a company that uses OPNSense firewall, so with the help of this tutorial (https://docs.opnsense.org/manual/how-tos/sslvpn_client.html) I configured an OpenVPN Server. It worked like a charm.

Now without anyone making any change it just stopped working, I can't connect to the openvpn Server. I already reconfigured the server, changed to TCP, restarted the firewall/daemon, turned off the firewall on the client side, experimented on another pc or network, but nothing, it just doesn't connect. Ohhh and I also updated OpenVPN GUI. Can you guys please help me with this? I just don't know what to do next, this is just wierd.

This is my config file:

dev tun
persist-tun
persist-key
cipher AES-256-CBC
auth SHA512
tls-client
client
resolv-retry infinite
remote 62.xx.xxx.xxx 1194 udp
lport 0
verify-x509-name "SSLVPN Server Certificate" name
pkcs12 100001402-CloudWall-udp-1194-xxxxxx.p12
tls-auth 100001402-CloudWall-udp-1194-xxxxxx-tls.key 1
ns-cert-type server
comp-lzo adaptive


Client log file:

Wed Mar 21 12:44:31 2018 OpenVPN 2.4.5 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] built on Mar  1 2018
Wed Mar 21 12:44:31 2018 Windows version 6.2 (Windows 8 or greater) 64bit
Wed Mar 21 12:44:31 2018 library versions: OpenSSL 1.1.0f  25 May 2017, LZO 2.10
Wed Mar 21 12:44:32 2018 WARNING: --ns-cert-type is DEPRECATED.  Use --remote-cert-tls instead.
Wed Mar 21 12:44:32 2018 TCP/UDP: Preserving recently used remote address: [AF_INET]62.28.222.252:1194
Wed Mar 21 12:44:32 2018 UDP link local (bound): [AF_INET][undef]:0
Wed Mar 21 12:44:32 2018 UDP link remote: [AF_INET]62.28.222.252:1194
Wed Mar 21 12:45:32 2018 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Wed Mar 21 12:45:32 2018 TLS Error: TLS handshake failed
Wed Mar 21 12:45:32 2018 SIGUSR1[soft,tls-error] received, process restarting


OpenVPN Log:
Mar 21 12:45:29   openvpn[2342]: MANAGEMENT: Client disconnected
Mar 21 12:45:29   openvpn[2342]: MANAGEMENT: CMD 'quit'
Mar 21 12:45:29   openvpn[2342]: MANAGEMENT: CMD 'status 2'
Mar 21 12:45:29   openvpn[2342]: MANAGEMENT: Client connected from /var/etc/openvpn/server1.sock
Mar 21 12:44:27   openvpn[2342]: MANAGEMENT: Client disconnected
Mar 21 12:44:27   openvpn[2342]: MANAGEMENT: CMD 'quit'
Mar 21 12:44:27   openvpn[2342]: MANAGEMENT: CMD 'status 2'
Mar 21 12:44:27   openvpn[2342]: MANAGEMENT: Client connected from /var/etc/openvpn/server1.sock
Mar 21 12:43:26   openvpn[2342]: MANAGEMENT: Client disconnected
Mar 21 12:43:26   openvpn[2342]: MANAGEMENT: CMD 'quit'
Mar 21 12:43:26   openvpn[2342]: MANAGEMENT: CMD 'status 2'
Mar 21 12:43:25   openvpn[2342]: MANAGEMENT: Client connected from /var/etc/openvpn/server1.sock
Mar 21 12:42:49   openvpn[2342]: MANAGEMENT: Client disconnected
Mar 21 12:42:49   openvpn[2342]: MANAGEMENT: CMD 'status 2'
Mar 21 12:42:49   openvpn[2342]: MANAGEMENT: Client connected from /var/etc/openvpn/server1.sock
Mar 21 12:42:24   openvpn[2342]: MANAGEMENT: Client disconnected.

[elektroinside]

Whenever I have this error, I usually recreate the CA, server certificate and client certificate as well. 99% of times it works and way faster than debugging the actual problem.

[s.messias]

Hello

Thank you so much for the suggestion, unfortunately, I'm the 1% :P

It didn't work

[elektroinside]

There is a very good tutorial on how to create the server here: https://docs.opnsense.org/manual/how-tos/sslvpn_client.html

Is it similar to your setup?

Sure thing, you're welcome!

[s.messias]

Thast's the tutorial I used to configure the server the first time, so yeah it's very similar. The only difference is that I use on server mode Remote Access (SSL/TLS). But I already tried to reconfigure with Remote Access (User Auth)and the result is the same =/

[elektroinside]

I have Remote Access (SSL/TLS + User Auth).

This is my server conf:

Code Select Expand


dev ovpns1
verb 0
dev-type tun
dev-node /dev/tun1
writepid /var/run/openvpn_server1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto tcp-server
cipher AES-256-CBC
auth SHA512
up /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup
down /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkdown
client-connect /usr/local/etc/inc/plugins.inc.d/openvpn/attributes.sh
client-disconnect /usr/local/etc/inc/plugins.inc.d/openvpn/attributes.sh
tls-server
server [edited] [edited]
client-config-dir /var/etc/openvpn-csc/1
username-as-common-name
auth-user-pass-verify "/usr/local/etc/inc/plugins.inc.d/openvpn/ovpn_auth_verify user 'Local Database' 'false' 'server1'" via-env
tls-verify "/usr/local/etc/inc/plugins.inc.d/openvpn/ovpn_auth_verify tls 'SSLVPN+Server+Certificate' 1"
lport 443
management /var/etc/openvpn/server1.sock unix
push "dhcp-option DOMAIN [edited]"
push "dhcp-option DNS [edited]"
push "register-dns"
push "dhcp-option NTP [edited]"
push "redirect-gateway def1"
client-to-client
duplicate-cn
route [edited] [edited]
ca /var/etc/openvpn/server1.ca
cert /var/etc/openvpn/server1.cert
key /var/etc/openvpn/server1.key
dh /usr/local/etc/dh-parameters.4096
tls-auth /var/etc/openvpn/server1.tls-auth 0
comp-lzo adaptive
topology subnet
reneg-sec 0
auth-nocache


For the client, on Windows, I use Viscosity and i exported the client config from the OPNsense GUI. Works fine for me.
For Android, I use OpenVPN connect. Also works fine.
Just saw some minor things i could probably optimize here, but nothing related to authentication.

[s.messias]

The wierd thing it's that it was working without any problem in the past month. Today when I tried to connect it, gave me the error, and we didn't make any change anywhere.
I'm going to analyse your configuration and try again a bunch of new configurations just to see the result :P

I will give a feedback when I have the time.

Thank u so much.

[elektroinside]

You could also try to update to OPNsense 18.1.5 having these updates (regarding OpenVPN):

o openvpn: switch status to version 3 to avoid wrong parsing of commas
o openvpn: parse all states to retrieve all relevant connection status info
o ports: openvpn 2.4.5[9]

Works fine for me so far (OpenVPN related stuff). I think i have some IPv6 issues, but unsure if it's because of the update.

[JayST]

today i got the same problem, only after upgrading to 18.1.9. It was working on 18.1.8!
i tried everything: recreate all certs, ca, openvpnserver etc.

From my openvpn client on windows:
TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
TLS Error: TLS handshake failed

[JayST]

hmmz this is weird. I got it working again.
Things i did to make it work:
1.) change the vpn server from udp to tcp and changed the firewall rules (wan and openvpn tabs) from udp to tcp too.
After that, it did not work yet.
2.) i tried to export the client configuration (as archive) again.
It still did not work.
3.) i noticed the client export procedure did not update the configuration file to reflect the new setting (tcp). It also kept the filename of the configuration zip file with "UDP" in it, even though it was now set to TCP.
4.) in the client export window, i changed "host name resolution" from the default "interface ip address" to "other" and then hardcoded my WAN address there
5.) this seems to trigger creation of a new config filename with upon exporting again. This time it had TCP in the filename and the ovpn file was reconfigured to use TCP.

I'm good :)