OpenVPN: remote routes work from shell, not from LAN

[sporkman]

I gave up on this before, but thought I'd try again with a fresh config on both ends.

Verified the server side is OK by connecting with a desktop client and verifying my certs are OK, that I can ping the remote OVPN interface and some IPs behind the VPN. All is well.

Also, if I ssh into the opnsense box, no problem. I can ping what I expect to be able to ping.

From the LAN though, my traffic all goes out the main WAN connection. Verified this with tcpdump.

Some quick examples follow...

From the shell:

Code: [Select]

root@SporkLab:/home/sporkadmin # ping 10.99.0.1
PING 10.99.0.1 (10.99.0.1): 56 data bytes
64 bytes from 10.99.0.1: icmp_seq=0 ttl=64 time=5.702 ms
64 bytes from 10.99.0.1: icmp_seq=1 ttl=64 time=7.859 ms

root@SporkLab:/home/sporkadmin # ping 10.88.77.72
PING 10.88.77.72 (10.88.77.72): 56 data bytes
64 bytes from 10.88.77.72: icmp_seq=0 ttl=64 time=7.829 ms
64 bytes from 10.88.77.72: icmp_seq=1 ttl=64 time=6.264 ms
When pinging from a host on the LAN, a tcpdump on the tun interface shows nothing:

Code: [Select]

frankentosh:2015-Hackintosh-Drive spork$ ping 10.99.0.1
PING 10.99.0.1 (10.99.0.1): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

root@SporkLab:/home/sporkadmin # tcpdump -vn -i ovpnc2 dst 10.99.0.1
tcpdump: listening on ovpnc2, link-type NULL (BSD loopback), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
But if I run tcpdump on the main WAN interface, I see what should be tunneled going right out the WAN interface:

Code: [Select]

root@SporkLab:/home/sporkadmin # tcpdump -vn -i re0 dst 10.99.0.1
tcpdump: listening on re0, link-type EN10MB (Ethernet), capture size 262144 bytes


14:01:34.515541 IP (tos 0x0, ttl 63, id 64433, offset 0, flags [none], proto ICMP (1), length 84)
    WAN IP > 10.99.0.1: ICMP echo request, id 5983, seq 0, length 64
14:01:35.516589 IP (tos 0x0, ttl 63, id 45486, offset 0, flags [none], proto ICMP (1), length 84)
    WAN IP > 10.99.0.1: ICMP echo request, id 5983, seq 1, length 64
14:01:36.516356 IP (tos 0x0, ttl 63, id 1206, offset 0, flags [none], proto ICMP (1), length 84)
    WAN IP > 10.99.0.1: ICMP echo request, id 5983, seq 2, length 64
I have no custom NAT rules for outbound.

I do have dual WAN setup as described in the docs.

Any idea what's happening?

[Maurice]

I do have dual WAN setup as described in the docs.

This means you use policy based routing, e. g. you have a firewall rule on the LAN interface which forces all packets coming from the LAN to your WAN gateway group. This overrides the routing table.

You need to add another, higher priority rule which allows packets going to your VPN network(s), but doesn't apply policy based routing to them.

Cheers

Maurice

[sporkman]

What would a rule like that look like? In my dropdown for destination, I've got "default", each individual physical gateway, null routes and the failover group, but not openvpn.

[Maurice]

A simple allow rule where you only specify the destination. The destination is the network(s) you want to exclude from being forcibly routed via the WAN gateway group. In your case: Networks which should be routed via VPN. You need to manually specify this (directly in the firewall rule or by creating an alias). If in doubt, have a look at your routing table.

Cheers

Maurice

[sporkman]

OK, so I added a rule with a bunch of aliases that represent the remote networks of interest. For gateway, left that at "default".

I still can't ping across the VPN, but the traffic is now visible on the firewall's tun interface:

Code: [Select]

root@SporkLab:~ # ifconfig ovpnc2
ovpnc2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
inet6 fe80::2e0:4cff:fe70:1219%ovpnc2 prefixlen 64 scopeid 0x9
inet 10.99.0.6 --> 10.99.0.5 netmask 0xffffffff
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
groups: tun openvpn
Opened by PID 70079
root@SporkLab:~ #
root@SporkLab:~ # tcpdump -nv -i ovpnc2 dst 10.99.0.1
tcpdump: listening on ovpnc2, link-type NULL (BSD loopback), capture size 262144 bytes
13:55:18.885355 IP (tos 0x0, ttl 63, id 32780, offset 0, flags [none], proto ICMP (1), length 84)
    10.3.2.40 > 10.99.0.1: ICMP echo request, id 27137, seq 0, length 64
13:55:19.888343 IP (tos 0x0, ttl 63, id 55650, offset 0, flags [none], proto ICMP (1), length 84)
    10.3.2.40 > 10.99.0.1: ICMP echo request, id 27137, seq 1, length 64
13:55:20.889000 IP (tos 0x0, ttl 63, id 22218, offset 0, flags [none], proto ICMP (1), length 84)
    10.3.2.40 > 10.99.0.1: ICMP echo request, id 27137, seq 2, length 64
13:55:21.889970 IP (tos 0x0, ttl 63, id 44700, offset 0, flags [none], proto ICMP (1), length 84)
    10.3.2.40 > 10.99.0.1: ICMP echo request, id 27137, seq 3, length 64
But it doesn't show up at the other end:

Code: [Select]

[root@foo-b /home/spork]# tcpdump -vn -i tun0
tcpdump: listening on tun0, link-type NULL (BSD loopback), capture size 262144 bytes


^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
[root@foo-b /home/spork]#
This is where ovpn's opaqueness trips me up. Why is it just eating the packets and not forwarding them on? The 10.99.0.1 IP should be an easy one, that's part of the tunnel subnet.

Of note, pinging 10.99.0.1 from the firewall's shell still works.

[Maurice]

Does the remote site have a route to the LAN on your site? So, when pinging 10.3.2.40 from 10.99.0.1, do these packets enter the tunnel?

Cheers

Maurice

[sporkman]

Finally!

On the server (a FreeBSD host), I had to do something to manipulate OpenVPN's internal routing.

In my "ccd" directory, I added a client-specific file for my opnsense box, with one line: "iroute 10.3.2.0 255.255.255.0", which if I understand this correctly, tells the openvpn process to shove packets it sees destined to that subnet to a specific connected client.

I then had to add a "route" statement to the server's openvpn.conf, "route 10.3.2.0 255.255.255.0". This adds a route (kernel) to the openvpn tun interface I think.

Neither really explains why my tcpdump wasn't showing the incoming traffic without that "iroute" statement, but I guess that remains a mystery.

Next up, what if I want to set some firewall rules on the opnsense openvpn client tunnel interface? This config works, but it also leaves my local LAN fully exposed to the remote network (I don't want that, I want outbound-only).