[SOLVED] OpenVPN - Captive portal user connection problem

[martain-vf]

Hey OPNsense community,

First of thank you for the this great piece of software. The simplicity of configuring the features and the clean interface is fantastic, makes me remember the good old m0n0wall days 

I hope someone can help me troubleshoot a issue i have between 2 OPNSense boxes...
One box is setup as OpenVPN appliance which is used for remote connection (road wariors) and the other one is a captive portal for our users and visitors...
Both boxes are deployed as a VM on Xenserver (with xen addon installed) and running the latest stable (atm 16.1.18-amd64)

The services work perfectly standalone but users on the captive portal arent't able to connect to the openVPN. One would think it would be a missing firewall rule but I sadly that's not it...   

Not sure if i still have some configuration problem, or are hitting a bug (on xenserver/opnsense/...) 

any help is greatly appreciated!

Below i have some notes and logging. It shows the Captive portal user gets trough to the OpenVPN box and the OpenVPN sends packets back to the Captive portal but i don't see the packets incoming on it's WAN interface...


Details:
Version: OPNsense 16.1.18-amd64 (on Xenserver)
Captive Portal ip: 1.2.3.4/26
OpenVPN ip: 1.2.3.5/26
Gateway ip: 1.2.3.1/26
^{(public ip's obfuscated)}

Schema:

               +---------+
               | Gateway |
               +----+----+
                    | 1.2.3.1/26
                    |
+---------+     WAN |      +---------+    LAN
| OpenVPN +---------+------+ Captive +---------
+---------+                +---------+  192.168.4.254/24
      1.2.3.5/26        1.2.3.4/26

Note: yes, the OpenVPN has only 1 interface

OpenVPN client log:

Code: [Select]

Wed Jul 13 17:58:39 2016 UDPv4 link local (bound): [undef]
Wed Jul 13 17:58:39 2016 UDPv4 link remote: [AF_INET]1.2.3.5:1194
Wed Jul 13 17:59:39 2016 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Wed Jul 13 17:59:39 2016 TLS Error: TLS handshake failed
Wed Jul 13 17:59:39 2016 SIGUSR1[soft,tls-error] received, process restarting
Wed Jul 13 17:59:41 2016 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info

pftop on Captive Portal box:

Code: [Select]

udp  In  192.168.4.12:1194  1.2.3.4:1194  NO_TRAFFIC:SINGLE  00:20:55  00:00:30  118  8260
udp  Out 1.2.3.5:2611       1.2.3.4:1194  SINGLE:NO_TRAFFIC  00:20:55  00:00:30  118  8260

pftop on OpenVPN box:

Code: [Select]

udp  In  1.2.3.5:2611       1.2.3.4:1194  MULTIPLE:MULTIPLE  00:27:58  00:00:53  505  36866

Capture on the captive portal box WAN (filter ipv4/udp 1.2.3.5:1194):

Code: [Select]

16:41:14.255174 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:16.400190 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:20.686570 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:21.413367 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:28.309389 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:28.578406 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:28.578420 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:33.072593 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:43.574568 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:44.945731 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:57.338775 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42

Capture on the OpenVPN box WAN (filter ipv4/udp 1.2.3.4:1194):

Code: [Select]

16:41:14.256074 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:14.256405 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 54
16:41:16.004197 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 42
16:41:16.400872 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:16.400948 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 50
16:41:20.026270 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 42
16:41:20.687263 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:20.687355 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 50
16:41:21.414221 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:21.414545 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 54
16:41:23.098900 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:28.000139 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 42
16:41:28.000173 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:28.310265 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:28.310347 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 50
16:41:28.579333 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:28.579342 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:28.579446 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 54
16:41:28.579504 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 50
16:41:30.041423 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:33.073281 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:33.073368 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 50
16:41:34.692590 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:36.003193 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:42.551212 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:43.575435 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:43.575525 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 50
16:41:44.016843 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 42
16:41:44.946658 IP 1.2.3.4.54857 > 1.2.3.5.1194: UDP, length 42
16:41:44.946755 IP 1.2.3.5.1194 > 1.2.3.4.54857: UDP, length 50
16:41:53.019427 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 42
16:41:57.339639 IP 1.2.3.4.2611 > 1.2.3.5.1194: UDP, length 42
16:41:57.339736 IP 1.2.3.5.1194 > 1.2.3.4.2611: UDP, length 50


[martain-vf]

[long outstanding update to my issue]

Strange enough it was a firewall (palo alto) om the same vlan that was interfering with the traffic...
after some additional allow rules on that firewall traffic was flowing steady