After upgrade to 18.1.8 OpenVPN site-to-site tunnel no longer comes up

[klausagnoletti]

Hi

After upgrading to 18.1.8 one of my OpenVPN site-to-site tunnels no longer comes up. On the connection status page in Opnsense, it's in status waiting:

Name   Remote Host   Virtual Addr   Connected Since   Bytes Sent   Bytes Received   Status   
Box Server VPN UDP:1194      10.100.100.1   2018-05-19 23:28:56   0 bytes   0 bytes   waiting   

Remote site is running Debian Linux. Remote networks are 10.20.40.0/24 and 172.40.172.0/24 - 10.100.100.1 is tunnel interface ip on fw, 10.100.100.2 is remote tunnel ip (on client side)

Opnsense wan ip is 10.49.141.2 (don't ask)


Tunnel is configured more or less like in the manual: https://wiki.opnsense.org/manual/how-tos/sslvpn_s2s.html

Opnsense Log:
May 19 23:50:20   openvpn[96107]: MANAGEMENT: Client disconnected
May 19 23:50:20   openvpn[96107]: MANAGEMENT: CMD 'state all'
May 19 23:50:20   openvpn[96107]: MANAGEMENT: Client connected from /var/etc/openvpn/server1.sock
May 19 23:50:14   openvpn[96107]: UDPv4 link remote: [AF_UNSPEC]
May 19 23:50:14   openvpn[96107]: UDPv4 link local (bound): [AF_INET]172.30.172.1:1194
May 19 23:50:14   openvpn[96107]: Socket Buffers: R=[42080->42080] S=[57344->57344]
May 19 23:50:14   openvpn[96107]: Could not determine IPv4/IPv6 protocol. Using AF_INET
May 19 23:50:14   openvpn[96107]: /sbin/route add -net 172.40.172.0 10.100.100.2 255.255.255.0
May 19 23:50:14   openvpn[96107]: ERROR: FreeBSD route add command failed: external program exited with error status: 1
May 19 23:50:14   openvpn[96107]: /sbin/route add -net 10.20.40.0 10.100.100.2 255.255.255.0
May 19 23:50:14   openvpn[96107]: /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup ovpns1 1500 1605 10.100.100.1 10.100.100.2 init
May 19 23:50:14   openvpn[96107]: /sbin/ifconfig ovpns1 10.100.100.1 10.100.100.2 mtu 1500 netmask 255.255.255.255 up
May 19 23:50:14   openvpn[96107]: do_ifconfig, tt->did_ifconfig_ipv6_setup=0
May 19 23:50:14   openvpn[96107]: TUN/TAP device /dev/tun1 opened
May 19 23:50:14   openvpn[96107]: TUN/TAP device ovpns1 exists previously, keep at program end
May 19 23:50:14   openvpn[96107]: ROUTE_GATEWAY 10.49.141.1/255.255.255.0 IFACE=vtnet0 HWADDR=86:5f:50:ed:2a:0e
May 19 23:50:14   openvpn[96107]: Incoming Static Key Encryption: Using 512 bit message hash 'SHA512' for HMAC authentication
May 19 23:50:14   openvpn[96107]: Incoming Static Key Encryption: Cipher 'AES-256-CBC' initialized with 256 bit key
May 19 23:50:14   openvpn[96107]: Outgoing Static Key Encryption: Using 512 bit message hash 'SHA512' for HMAC authentication
May 19 23:50:14   openvpn[96107]: Outgoing Static Key Encryption: Cipher 'AES-256-CBC' initialized with 256 bit key
May 19 23:50:14   openvpn[96107]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
May 19 23:50:14   openvpn[96107]: MANAGEMENT: unix domain socket listening on /var/etc/openvpn/server1.sock
May 19 23:50:14   openvpn[95869]: library versions: LibreSSL 2.6.4, LZO 2.10
May 19 23:50:14   openvpn[95869]: OpenVPN 2.4.6 amd64-portbld-freebsd11.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on May 16 2018

Client log reveals absolutely nothing:
May 19 23:57:06 box ovpn-fw-udp-1194[3138]: disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
May 19 23:57:06 box ovpn-fw-udp-1194[3138]: WARNING: file 'fw-udp-1194.secret' is group or others accessible
May 19 23:57:06 box ovpn-fw-udp-1194[3138]: OpenVPN 2.4.0 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 18 2017
May 19 23:57:06 box ovpn-fw-udp-1194[3138]: library versions: OpenSSL 1.0.2l  25 May 2017, LZO 2.08
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: TUN/TAP device tun0 opened
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: do_ifconfig, tt->did_ifconfig_ipv6_setup=0
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: /sbin/ip link set dev tun0 up mtu 1500
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: /sbin/ip addr add dev tun0 local 10.100.100.2 peer 10.100.100.1
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: TCP/UDP: Preserving recently used remote address: [AF_INET]10.49.141.2:1194
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: UDP link local (bound): [AF_INET][undef]:1194
May 19 23:57:06 box ovpn-fw-udp-1194[3139]: UDP link remote: [AF_INET]10.49.141.2:1194

In all honesty, I have no idea WTF is goiong on since literally the only change I have done in this setup is the Opnsense update. The only obvious error is the route add that fails. It's weird that it does - and I don't get if that should prevent the tunnel from coming up. I can't ping any devices on either of the remote networks.

Can anyone please help me before I go nuts?

Thanks,

/klaus

[klausagnoletti]

Argh

Can it really be true that noone here has any kind of indicators as to what can be wrong? I have no idea what so ever myself. That makes debugging more or less impossible, so any comments at all would help me. Please.

Thanks!

/Klaus