OPNsense Forum

Archive => 23.7 Legacy Series => Topic started by: guest42255 on January 25, 2024, 01:29:52 pm

Title: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: guest42255 on January 25, 2024, 01:29:52 pm
Hello,

I need an OpenVPN Site-to-Site VPN from my OPNsense 23.7 box to a Ubiquiti EdgeRouter.
After configuring everything, just nothing works without any reason. There are no useful error messages.

The OPNsense config is as following:
Server ModePeer to Peer (Shared Key)
ProtocolUDP
Device Modetun
InterfaceWAN
Local Port1195
Encryption algorithmAES-256-CBC
Auth Digest AlgorithmSHA512
Shared Key2048 bit OpenVPN static key
IPv4 Tunnel Network10.255.0.1/30
IPv4 Local Network172.16.140.0/24
IPv4 Remote Network172.16.1.0/24,10.0.0.0/16
Concurrent connections1
CompressionNo Preference
Topologychecked
Verbosity level11

All not mentioned settings are empty/unchecked.

The following messages are shown in the log:

The EdgeRouter (latest firmware) configuration is:

Code: [Select]
openvpn vtun0 {
    encryption aes256
    hash sha512
    local-address 10.255.0.2 {
    }
    local-port 1195
    mode site-to-site
    remote-address 10.255.0.1
    remote-host ********** (IP of OPNsense WAN interface)
    remote-port 1195
    shared-secret-key-file /config/auth/secret
}

The log output on the EdgeRouter is totally useless as well:
Code: [Select]
disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
WARNING: file '/config/auth/secret' is group or others accessible
OpenVPN 2.4.7 mips-unknown-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Apr 22 2022
library versions: OpenSSL 1.0.2u  20 Dec 2019, LZO 2.08
Outgoing Static Key Encryption: Cipher 'AES-256-CBC' initialized with 256 bit key
Outgoing Static Key Encryption: Using 512 bit message hash 'SHA512' for HMAC authentication
Incoming Static Key Encryption: Cipher 'AES-256-CBC' initialized with 256 bit key
Incoming Static Key Encryption: Using 512 bit message hash 'SHA512' for HMAC authentication
TUN/TAP device vtun0 opened
TUN/TAP TX queue length set to 100
/sbin/ip link set dev vtun0 up mtu 1500
/sbin/ip addr add dev vtun0 local 10.255.0.2 peer 10.255.0.1
TCP/UDP: Preserving recently used remote address: [AF_INET]**********:1195
Socket Buffers: R=[294912->294912] S=[294912->294912]
UDP link local (bound): [AF_INET][undef]:1195
UDP link remote: [AF_INET]**********:1195
/usr/bin/vtysh-set -c configure terminal -c router rip -c network vtun0 failed: 10752
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: tkrn on January 25, 2024, 01:37:29 pm
The root of the problem is that Ubiquiti is using an old version of OpenVPN (2.4.7) and OPNsense is using 2.6.8 which changes security significantly. I have the same exact issue with an existing site-to-site but that client can only do AES-256 (AES-256-CBC) at best. WIth that being said, once I increase the log level on the OpenVPN instance on OPNsesne it turns out that the root of the issue is the cipher not being able to exchange keys

Heres the version information from an ER-4 on v2.0.9-hotfix 9:
Code: [Select]
OpenVPN 2.4.7 mips-unknown-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Apr 22 2022
library versions: OpenSSL 1.0.2u  20 Dec 2019, LZO 2.08
Originally developed by James Yonan
Copyright (C) 2002-2018 OpenVPN Inc <sales@openvpn.net>
Compile time defines: enable_async_push=no enable_comp_stub=no enable_crypto=yes enable_crypto_ofb_cfb=yes enable_debug=yes enable_def_auth=yes enable_dependency_tracking=no enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=needless enable_fragment=yes enable_iproute2=yes enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_maintainer_mode=no enable_management=yes enable_multihome=yes enable_pam_dlopen=no enable_pedantic=no enable_pf=yes enable_pkcs11=yes enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_server=yes enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=no enable_strict_options=no enable_systemd=yes enable_werror=no enable_win32_dll=yes enable_x509_alt_username=yes with_aix_soname=aix with_crypto_library=openssl with_gnu_ld=yes with_mem_check=no with_sysroot=no
The OPNsense log points to this:
Code: [Select]
PUSH: No common cipher between server and client. Server data-ciphers: 'AES-256-GCM', client supports cipher 'AES-256-CBC'
After some more investigation, the cipher list in EdgeOS does support AES-256-GCM:
Code: [Select]
/usr/sbin/openvpn --show-ciphers
The following ciphers and cipher modes are available for use
with OpenVPN.  Each cipher shown below may be use as a
parameter to the --cipher option.  The default key size is
shown as well as whether or not it can be changed with the
--keysize directive.  Using a CBC or GCM mode is recommended.
In static key mode only CBC mode is allowed.

AES-128-CBC  (128 bit key, 128 bit block)
AES-128-CFB  (128 bit key, 128 bit block, TLS client/server mode only)
AES-128-CFB1  (128 bit key, 128 bit block, TLS client/server mode only)
AES-128-CFB8  (128 bit key, 128 bit block, TLS client/server mode only)
AES-128-GCM  (128 bit key, 128 bit block, TLS client/server mode only)
AES-128-OFB  (128 bit key, 128 bit block, TLS client/server mode only)
AES-192-CBC  (192 bit key, 128 bit block)
AES-192-CFB  (192 bit key, 128 bit block, TLS client/server mode only)
AES-192-CFB1  (192 bit key, 128 bit block, TLS client/server mode only)
AES-192-CFB8  (192 bit key, 128 bit block, TLS client/server mode only)
AES-192-GCM  (192 bit key, 128 bit block, TLS client/server mode only)
AES-192-OFB  (192 bit key, 128 bit block, TLS client/server mode only)
AES-256-CBC  (256 bit key, 128 bit block)
AES-256-CFB  (256 bit key, 128 bit block, TLS client/server mode only)
AES-256-CFB1  (256 bit key, 128 bit block, TLS client/server mode only)
AES-256-CFB8  (256 bit key, 128 bit block, TLS client/server mode only)
AES-256-GCM  (256 bit key, 128 bit block, TLS client/server mode only)
AES-256-OFB  (256 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-128-CBC  (128 bit key, 128 bit block)
CAMELLIA-128-CFB  (128 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-128-CFB1  (128 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-128-CFB8  (128 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-128-OFB  (128 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-192-CBC  (192 bit key, 128 bit block)
CAMELLIA-192-CFB  (192 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-192-CFB1  (192 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-192-CFB8  (192 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-192-OFB  (192 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-256-CBC  (256 bit key, 128 bit block)
CAMELLIA-256-CFB  (256 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-256-CFB1  (256 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-256-CFB8  (256 bit key, 128 bit block, TLS client/server mode only)
CAMELLIA-256-OFB  (256 bit key, 128 bit block, TLS client/server mode only)
SEED-CBC  (128 bit key, 128 bit block)
SEED-CFB  (128 bit key, 128 bit block, TLS client/server mode only)
SEED-OFB  (128 bit key, 128 bit block, TLS client/server mode only)

The following ciphers have a block size of less than 128 bits,
and are therefore deprecated.  Do not use unless you have to.

BF-CBC  (128 bit key by default, 64 bit block)
BF-CFB  (128 bit key by default, 64 bit block, TLS client/server mode only)
BF-OFB  (128 bit key by default, 64 bit block, TLS client/server mode only)
CAST5-CBC  (128 bit key by default, 64 bit block)
CAST5-CFB  (128 bit key by default, 64 bit block, TLS client/server mode only)
CAST5-OFB  (128 bit key by default, 64 bit block, TLS client/server mode only)
DES-CBC  (64 bit key, 64 bit block)
DES-CFB  (64 bit key, 64 bit block, TLS client/server mode only)
DES-CFB1  (64 bit key, 64 bit block, TLS client/server mode only)
DES-CFB8  (64 bit key, 64 bit block, TLS client/server mode only)
DES-EDE-CBC  (128 bit key, 64 bit block)
DES-EDE-CFB  (128 bit key, 64 bit block, TLS client/server mode only)
DES-EDE-OFB  (128 bit key, 64 bit block, TLS client/server mode only)
DES-EDE3-CBC  (192 bit key, 64 bit block)
DES-EDE3-CFB  (192 bit key, 64 bit block, TLS client/server mode only)
DES-EDE3-CFB1  (192 bit key, 64 bit block, TLS client/server mode only)
DES-EDE3-CFB8  (192 bit key, 64 bit block, TLS client/server mode only)
DES-EDE3-OFB  (192 bit key, 64 bit block, TLS client/server mode only)
DES-OFB  (64 bit key, 64 bit block, TLS client/server mode only)
DESX-CBC  (192 bit key, 64 bit block)
RC2-40-CBC  (40 bit key by default, 64 bit block)
RC2-64-CBC  (64 bit key by default, 64 bit block)
RC2-CBC  (128 bit key by default, 64 bit block)
RC2-CFB  (128 bit key by default, 64 bit block, TLS client/server mode only)
RC2-OFB  (128 bit key by default, 64 bit block, TLS client/server mode only)

It turns out that it will have to be a custom configuration on the Ubquiti side which you'll need to use a traditional ovpn profile and set EdgeOS to use it like:
Code: [Select]
configure
set interfaces openvpn vtun0 config-file /config/site-to-site.ovpn
commit

I'm going to give that a try this cipher list a try on my EdgeRouter and I'll let you know how it goes soon:
Code: [Select]
data-ciphers AES-256-GCM:AES-128-GCM:AES-256-CBC
data-ciphers-fallback AES-256-CBC

I believe this to be the problem because in the EdgeOS interface when configuring site to site it's assumed that it's "In static key mode" which means only "CBC mode is allowed." The documentation and interface doesn't use of GCM in site-to-site mode.  I *believe* that is the core issue thus you have to fall back to a traditional OpenVPN profile.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: guest42255 on January 25, 2024, 02:07:14 pm
I am already on the highest possible log level and I don't get any of those messages. I have both the EdgeRouter and OPNsense supporting AES-256-CBC. The static key was also generated on the EdgeRouter.

As far as I can tell, the OpenVPN daemon on the OPNsense box doesn't even start because it doesn't care about the configured routes and therefore fails to launch.

For your issue I would just set the OPNsense part to use AES-256-CBC because it's supported on both routers.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: tkrn on January 25, 2024, 08:50:31 pm
I decided to write up a blog article on this topic since it has a lot of moving parts. Here take a look, let me know if this all makes sense to you. It makes sense to me but I wanted to confirm on your end.

https://blog.tkrn.io/site-to-site-openvpn-between-opnsense-and-ubiquiti-edgerouter-edgeos/
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: guest42255 on January 26, 2024, 11:06:39 pm
Unfortunately that doesn't help me as I don't have mismatching ciphers. My ciphers match but there is a bug in OPNsense which makes it impossible to start the OpenVPN adapter. This bug can be reproduced with any fresh install of OPNsense independently of what the other peer would be. Just configure a Peer-to-Peer VPN and it will immediately break and the adapter won't even start.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: Fright on January 27, 2024, 07:54:58 pm
why does everyone love /30 subnet so much? )
can you try to use /29 tunnel subnet if you use "Topology" (=='--topology subnet')?
or dont use "Topology" if you want /30 tunnel subnet?
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: guest42255 on January 28, 2024, 11:42:27 am
I have found the solution to the issue. The exact description of what causes the bug is found on GitHub: https://github.com/opnsense/core/issues/7167#issuecomment-1913536195

So the subnet-topology is broken. The ifconfig command is used incorrectly in the resulting config file. So "Topology" is unusable. Just unchecking it results in the tunnel to work.

I have tested it with a /29 subnet and it also doesn't work. Now knowing what causes the bug makes this obvious.

As long as this bug is there a point-to-point tunnel is possible but a point-to-multipoint is impossible. But for my case I don't need that anyways.

Unfortunately it is refused to see this obvious bug and so it will never be fixed. Also I have heard out that OpenVPN Site-to-Site could be entirely discontinued by OPNsense in the future.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: Fright on January 28, 2024, 01:37:54 pm
Quote
I have tested it with a /29 subnet and it also doesn't work
i think you wrong. with /29 and Topology enabled server starts just fine
Quote
Unfortunately it is refused to see this obvious bug
IMHO I think Ad explained everything in the github thread.
If you really want to make product better and not just mention "bug" twice per post, sorry, its better to use Instances (not 'Servers' menu. i think e.g. Instance form would not allow /30 net with the subnet topo) and try to share comprehensive info (configuration, errors from logs with debugging enabled): as far as i can see Ad is migrating OpenVPN to MVC and it doesn’t make much sense to improve the legacy ("Servers") page.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: guest42255 on January 29, 2024, 09:11:09 pm
I got with a /29 subnetz exactly the same error messages.

My initial ticket contained all steps to reproduce and all log output when debug logs were enabled. There was just almost no log output, even when using debug logging.

The "Instances" menu doesn't help me, because you can there only create server and client modes, but not the site-to-site mode of OpenVPN. So this wouldn't be an equivalent solution which would result in completely restructuring how all of our locations communicate with each other.

I'm sorry that I won't call a very obvious bug something else, just so nobody feels offended. I don't know who or when this bug was introduced, I was just pointing out what the exact cause of it is.

I will not discuss any further that "this is not a bug", because it clearly is and it is reproducible.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: Patrick M. Hausen on January 29, 2024, 09:21:19 pm
Then please do open a bug ticket. This is the community forum - users helping other users. If you want attention by developers and have reason to think there is a serious bug, then use the right platform. Bug issues are on Github.
Title: Re: OpenVPN Site-to-Site with Ubiquiti EdgeRouter not working
Post by: Fright on January 30, 2024, 07:43:12 am
Quote
all log output when debug logs were enabled
Quote
The "Instances" menu doesn't help me, because you can there only create server and client modes, but not the site-to-site mode of OpenVPN.
Quote
I will not discuss any further
oh... I agree, sorry: seems like a waste of time.
Good luck!