Configuring a routed IPSec tunnel breaks connectivity

[gjarboni]

Hello,

I have a bit of an oddball situation. I'm using OPNSense (v19.7) as a router for IPSec VTI connections (or route-based tunnels), nothing else. I can't be on the public Internet due to restrictions at the site. So one physical network interface and, currently, one routed IPSec tunnel. The remote end is a Cisco router running 15.1-4M10 directly connected to the Internet.

Now, this is the strange part. On two separate firewalls enabling this set up has caused the firewall to copmletely lose connectiity. The OPNSense box isn't even getting ARP replies. Anyway, since this happened twice, I'm reasonably confident it's something I'm doing and not hardware related. BTW, the "hardware" for these boxes is virtual under VMWare. One box is using the VMWare NIC and the other is using e1000 emulation (em0)

I'm thinking this is related to NAT, but I'm not sure. Running pfctl -d doesn't help, though (I don't know if that makes sense or not).

Anyway, I'm hoping someone can give me some insight. Thanks in advance!

Jason M.

[mimugmail]

19.7 or 19.7.3?

[gjarboni]

The console says 19.7 (no .3). Is that enough or should I look elsewhere? Thanks.

[mimugmail]

Please update to .3

[gjarboni]

Okay, I've updated and no change. This is what I did.

Reset to factory default (so I could access the network again)
Ran opnsense-update -- rebooted when done
Copied backup of config to /conf/config.xml
Rebooted

The result is the same -- no network connectivity. I also redid the setup this time remembering to disable NAT, and added an additional interface for the WAN (even though it's not on the Internet). Still the same problem. When I finish the configuration for the routed IPSec tunnel, all network connectivity is lost.

Any ideas?

[gjarboni]

This probably won't surprise anyone but running:

/etc/rc.d/ipsec onestop

restores connectivity to the box. However once I did this, I could access the LAN, but couldn't ping anything reachable via the WAN. I previously put in rules to pass any traffic on LAN and WAN (for testing) and turned off NAT.

[gjarboni]

Okay, this is even more curious. After disabling IPSec and SSHing to the OPNSense box I see the routing table:

root@OPNsense2:~ # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
0.0.0.0/1          A.B.C.1          US         vmx0
default            A.B.C.1          UGS        vmx0
A.B.C.0/24         link#1             U          vmx0
A.B.C.3            link#1             UHS         lo0
G.H.I.195          A.B.C.1          UGHS       vmx0
G.H.I.198          A.B.C.1          UGHS       vmx0
127.0.0.1          link#3             UH          lo0
128.0.0.0/1        A.B.C.1          US         vmx0
D.E.23.1           link#7             UHS         lo0
D.E.23.2           link#7             UH     ipsec100
D.E.F.0/24         link#2             U           em0
D.E.F.3            link#2             UHS         lo0

I'm focusing on the 0.0.0.0/1 and 128.0.0.0/1 routes. They look odd so I delete them:

root@OPNsense2:~ # route delete -net 0.0.0.0/1 A.B.C.1
delete net 0.0.0.0: gateway A.B.C.1
root@OPNsense2:~ # route delete -net 128.0.0.0/1 A.B.C.1
delete net 128.0.0.0: gateway A.B.C.1
root@OPNsense2:~ # netstat -rn | more
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            A.B.C.1          UGS        vmx0
A.B.C.0/24         link#1             U          vmx0
A.B.C.3            link#1             UHS         lo0
G.H.I.195          A.B.C.1          UGHS       vmx0
G.H.I.198          A.B.C.1          UGHS       vmx0
127.0.0.1          link#3             UH          lo0
D.E.23.1           link#7             UHS         lo0
D.E.23.2           link#7             UH     ipsec100
D.E.F.0/24         link#2             U           em0
D.E.F.3            link#2             UHS         lo0

And now I can ping via the WAN:

root@OPNsense2:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=53 time=13.144 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=53 time=13.146 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=53 time=13.238 ms

Any idea where those routes came from? My home OPNSense box doesn't have them. I'm mystified.

[gjarboni]

One more data point. Killing charon restores connectivity as well.

[gjarboni]

Okay, it turns out it was the "Install policy" setting that was breaking things. Thanks.

[mimugmail]

Great! But did you set it? It's stated to not set it :)

[gjarboni]

Unforatunely, it's enabled by default. I don't remember setting it, but I could have just forgotten. Not so nice to have a "history erasing" button in the gui :)

[geotek]

I fell into the same ditch as you until I found out via this very helpful post that "Install Policy" must be unchecked. I hope this behaviour will be changed in the future because simply selecting "Route Based" with the default settings must not render the whole firewall unreachable.