OpenVPN Site-to-Site with SSL/TLS not working. Ping works but nothing else

[d39FAPH7]

Hello,
i am migrating all my routers from pfSense to OPNsense. So far i am quite happy with it but OpenVPN S2S with Certificates does not work. It is a setup i've been using for many years and basically follows this guide:

https://docs.netgate.com/pfsense/en/latest/recipes/openvpn-s2s-tls.html
I also read this guide:
https://docs.opnsense.org/manual/how-tos/sslvpn_s2s.html
and also tried the new "Instances" feature following this guide:
https://docs.opnsense.org/manual/how-tos/sslvpn_instance_s2s.html

Essentially these guides follow the same logic. "Instances" has a bit less options.

I have two sites. One server and one client. OPNsense version is OPNsense 23.7.6-amd64 on both sites. Connection can be established. Everything looks good. Routes seem correct to me. I can ping machines from client site located at server site but cannot reach webinterfaces in browser or anything else located at server site.

I call the server site "headquarter" and the client site "warehouse" in this example.
headquarter local net is 10.0.16.0/21 and warehouse local net is 10.0.48.0/21

This is the config:

Server site certs setup

System
   Trust
      Certificates
      "+Add"
         Method: Create an internal certificate
         Descriptive name: openvpn_s2s_routesubnet:headquarter-opnsense.allmysites.de
         Internal Certificate
            Certificate authority: headquarter-opnsense.allmysites.de
            Certificate Type: Server certificate
            Key type: RSA
            Key length: 2048 (default)
            Digest Algorithm: sha256 (default)
            Lifetime (days): 3650
            Private key location: Save on this firewall (default)
         Distinguished name
            Country Code: DE
            State or Province: headquarter-opnsense
            City: headquarter-opnsense
            Organization: headquarter-opnsense
            Email Address: headquarter-opnsense
            Common Name: headquarter-opnsense.allmysites.de
            Alternative Names:
               Type: DNS
               Value: headquarter-opnsense.allmysites.de
      => Save

      Certificates
      +Add/Sign
         Method: Create an internal certificate
         Descriptive name: openvpn_s2s_routesubnet:headquarter-opnsense.allmysites.de:warehouse.allmysites.de
         Internal Certificate
            Certificate authority: headquarter-opnsense.allmysites.de
            Certificate Type: Client certificate
            Key type: RSA
            Key length: 2048 (default)
            Digest Algorithm: sha256 (default)
            Lifetime (days): 3650
            Private key location: Save on this firewall (default)
         Distinguished name
            Country Code: DE
            State or Province: headquarter-opnsense
            City: headquarter-opnsense
            Organization: headquarter-opnsense
            Email Address: headquarter-opnsense
            Common Name: warehouse.allmysites.de
            Alternative Names:
               Type: DNS
               Value: warehouse.allmysites.de
      => Save

OpenVPN Server config

VPN
   OpenVPN
      Servers
      "+Add"
         General Information
            Disabled: unchecked
            Description: headquarter-opnsense.mysites.de (OpenVPN, Site-to-Site, Route only Subnet)
            Server mode: Peer to Peer (SSL/TLS): selected (default)
            Protocol: UDP4: selected (default)
            Device mode: tun:selected (default)
            Interface: any: selected (default)
            Local port: 12345
         Cryptographic Settings
            TLS Authentication: Enabled - Authentication only: selected (default)
            Automatically generate a TLS Key: checked (default)
            Peer Certificate Authority: headquarter-opnsense.mysites.de
            Peer Certificate Revocation List: None: selected (default)
            Server certificate: openvpn_s2s_routesubnet:headquarter-opnsense.mysites.de: selected
            Encryption algorithm (deprecated): AES-256-CBC (256 bit key, 128-bit block): selected
            Auth Digest Algorithm: SHA256 (256-bit): selected
            Certificate Depth: One (Client+Server) (default)
         Tunnel Settings
            IPv4 Tunnel Network: 10.0.25.0/24
            IPv6 Tunnel Network: empty (default)
            Redirect Gateway: unchecked (default)
            IPv4 Local network: 10.0.16.0/21
            IPv6 Local network:
            IPv4 Remote network: 10.0.48.0/21
            IPv6 Remote network: empty (default)
            Concurrent connections - empty (default)
            Compression: Legacy - Disabled LZO algorithm (--comp lzo no): selected
            Type-of-Service: unchecked (default)
            Duplicate Connections: unchecked (default)
         Client Settings
            Dynamic IP: unchecked (default)
            Topology: unchecked (default)
            Client Management Port: unchecked (default)
         Advanced Configuration
            Verbosity level: 3 (recommended): selected
            Force CSO Login Matching: unchecked (default)

      Client Specific Overrides
      "+Add"
         General Information
            Disabled: unchecked (default)
            Servers: Description: headquarter-opnsense.mysites.de (OpenVPN, Site-to-Site, Route only Subnet) (12345 / UDP4)
            Description: empty (default)
            Common name: warehouse-opnsense.mysites.de
            Connection blocking: unchecked (default)
         Tunnel Settings
            IPv4 Tunnel Network: empty (default)
            IPv6 Tunnel Network: empty (default)
            IPv4 Local Network: 10.0.16.0/21
            IPv4 Remote Network: 10.0.48.0/21
            Redirect Gateway: Nothing selected (default): selected
            => Save

Firewall
   Rules
      WAN
      "+Add"
         Interface: WAN: selected
         Direction: in: selected
         TCP/IP Version: IPv4: selected
         Protocol: UDP: selected
         Source: any: selected
         Destination: WAN address: selected
         Destination port range
            From: other: Selected
               Custom: 12345
            To: other: Selected
               Custom: 12345
         Description: OpenVPN
      => Save => Apply changes

      OpenVPN
      "+Add"
         Interface: OpenVPN: selected
         Direction: in: selected
         TCP/IP Version: IPv4: selected
         Protocol: any: selected
         Source: any: selected
         Destination: any: selected
         Description: OpenVPN
      => Save => Apply changes

Client site certs setup

System
   Trust
      Authorities
      "+Add"
         Descriptive name: headquarter-opnsense.mysites.de
         Method: Import an existing Certificate Authority
         Existing Certificate Authority
            Certificate data: Paste a certificate in X.509 PEM format here. (Export the data from headquarter-opnsense and open it in a texteditor for copypaste)
            Certificate Private Key (optional): empty (default)
            Serial for next certificate: empty (default)
      => Save

      Certificates
      +Add
         Method: Import an existing Certificate
         Descriptive name: openvpn_s2s_routesubnet:headquarter-opnsense.mysites.de:warehouse-opnsense.mysites.de
         Import Certificate
            Certificate data: Paste a certificate in X.509 PEM format here. (Export the data from headquarter-opnsense and open it in a texteditor for copypaste)
            Private key data: Paste a certificate in X.509 PEM format here. (Export the data from headquarter-opnsense and open it in a texteditor for copypaste)
      => Save

OpenVPN Client config

VPN
   OpenVPN
      Clients
      +Add
         General information
            Disabled: unchecked
            Description: headquarter-opnsense.mysites.de (OpenVPN, Site-to-Site, Route only Subnet)
            Server mode: Peer to Peer (SSL/TLS): selected (default)
            Protocol: UDP4: selected (default)
            Device mode: tun: selected (default)
            Interface: any: selected (default)
            Remote server
               Server host or address: headquarter-opnsense.mysites.de
               Port: 12345
               Select remote server at random: unchecked (default)
            Retry DNS resolution - Infinitely resolve server: checked
            Proxy host or address: empty (default)
            Proxy port: empty (default)
            Proxy authentication extra options: none: selected (default)
            Local port: 0

         User Authentication Settings
            Username: empty (default)
            Password: empty (default)
            Renegotiate time: empty (default)
         Cryptographic Settings
            TLS authentication: Enabled - Authentication only: selected (default)
            Automatically generate a shared TLS authentication key: unchecked
               Key: Paste the shared key here (copypaste from headquarter-opnsense OpenVPN Server config page)
            Peer Certificate Authority: headquarter-opnsense.mysites.de: selected
            Client certificate: openvpn_s2s_routesubnet:headquarter-opnsense.mysites.de:warehouse-opnsense.mysites.de (CA: headquarter-opnsense.mysites.de): selected
            Encryption Algorithm: AES-256-CBC (256bit, 128bit block): selected
            Auth digest algorithm: SHA256 (256bit): selected
         Tunnel Settings
            IPv4 Tunnel Network: 10.0.25.0/24
            IPv6 Tunnel Network: empty (default)
            IPv4 Remote network(s): 10.0.16.0/21
            IPv6 Remote network(s): empty (default)
            Limit outgoing bandwidth: empty (default)
            Compression: Compression: Legacy - Disabled LZO algorithm (--comp lzo no): selected
            Type-of-Service: unchecked (default)
            Don't pull routes: unchecked (default)
            Don't add/remove routes: unchecked (default)
         Advanced Configuration
            Advanced: empty
            Verbosity level: 3 (recommended): selected
      => Save

Firewall
   Rules
      OpenVPN
      "+Add"
         Interface: OpenVPN: selected
         Direction: in: selected
         TCP/IP Version: IPv4: selected
         Protocol: any: selected
         Source: any: selected
         Destination: any: selected
         Description: OpenVPN
      => Save => Apply changes

[bartjsmit]

I've had this and fixed it by adjusting the MTU

[d39FAPH7]

OK, you are a genius. One site is DSL (1492) and one Cable (1500). I've changed MTU in OpenVPN settings and boom it works.
This should be mentioned in guides as it will save you from a headache.

I did set MTU in my older pfSense configs but there it was under "Advanced configuration" which now in OPNsense "will be removed in the future due to being insecure by nature". This was the reason i cancelled that setting. Good that this is now a regular option in OPNsense.