IPSEC/routing issue not bringing up tunnel

[bickns]

Greetings - I'm new here.  I've just spent the past two days trying to migrate a Watchguard T40 onto a Dell Wyse 5070 OPNSense device .. it all started ok, but then yesterday thinking I had gotten it all working I started having issues with the BOVPN.

The other end is Watchguard and unsurprisingly the T40 and WG managed things nicely.    I originally had some issues with activity timeouts, which I think I have sorted but the big issue is that OPNSense doesn't always want to bring up a tunnel - so if I try and ping a machine on the other end of the WG, I would normally expect the box to bring up the tunnel based on the IP address, but it just (sometimes) tries to send the ping across the WAN.   I set the ping to continuous, and then tell the WG to rekey the tunnel and this then fixes the issue.  The intermittent nature of the problem might indicate that things only work when the WG is the initiator.   

I've checked the WAN option to Block Private Networks in the hope this would cajole the OPNSense into checking the IPSEC routing table before sending the ping over the WAN but still no joy.   I'm using the legacy Tunnel Settings /config (couldn't get my head round the new connections options, too much time spent with Phase 1 and Phase 2 etc) - so wondering if there is something obvious I am missing to get the auto tunnel creation to work on routing traffic ?

I'm just about to investigate whether I can do this with Wireshark instead, but hoped someone might be able to shed some light on this.

All help appreciated

Cheers

[bickns]

By way of extra info .. So I just noticed that all the automatically generated/installed security policy database entries all seem to have the same reqid (the first entry) - which is wrong (I have 6 tunnels for one phase 1). 

So I tried adding Manual SPD entries for each of the tunnels and this hasn't changed things.  Also the Security Association database has all the same reqid=1 (which is wrong) - so I'm just wondering if the system defaults (i.e only works) when there is one tunnel per phase 1 and I'm going to have to setup a phase one for each tunnel if this is ever going to work ??

[netnut]

...
so I'm just wondering if the system defaults (i.e only works) when there is one tunnel per phase 1 and I'm going to have to setup a phase one for each tunnel if this is ever going to work ??

A single Phase 1 with multiple Phase 2 (six in your case) shouldn't be a problem from a OPNsense (strongswan) point of view. Your issues sounds quite like Fortinet/Fortigate quirks - https://docs.strongswan.org/docs/5.9/interop/fortinet.html#_other_known_quirks

You describe multiple issues, you might want to start with a single Phase 1 / Phase 2 and sort things out before adding more complexity (ie. more Phase 2's). I highly recommend trying the new connection based setup (again) on OPNsense, it gives you way more control over the IKE behavior (setting req id's, trap policies, etc.). Exactly the items you're reporting issues with...

Setup your connection, be sure your trap policies setup do work to initiate automatic connection, be sure the rekeying is working from both sides (timers!). Only if you can do _all_ of this successfully for a single phase 2, add more phase 2's, otherwise debugging gets really messy and issues may overlap.

[bickns]

Thanks netnut - i did some more diagnostics last night and can confirm that it definitely didn't work with 6 tunnels off one phase one.  The difference might be that I have 3 distinct subnets on the OPNsense end and the some of the tunnels have the same endpoint subnet - this materialised with the OPNsense not using the existing tunnel (where it eventually times out due to inactivity) when I bring up another tunnel with the same destination subnet.

In the end I decided to stack my 3 local subnets into a /22 and then just configured one tunnel with the plan being to use rules to restrict traffic instead of relying on routing to send stuff down the correct tunnel.  The tunnel has been stable for the last 6 1/2hrs and its due to expire in 90mins so will see if this gets handled correctly. So looks like I've got a working solution.

The more worrying thing is that the OPNsense was trying to send stuff across the WAN instead of bringing up a tunnel - but this could be its inaccurate security policy database not working.

The other thing was when I finally got to get some firewalling out of it I had to add something similar to "Default allow LAN to any rule" for any interface to work properly, but this can't be right as it seems to allow inbound traffic  back to the interface.  I would normally expect to configure the outbound rule and the fw will know to return associated packets without having an explicit inbound rule ... However this likely means that I don't fully understand how it does things yet - so will try and find the part of the documentation which explains how the rules work compared to my experience with other firewalls.

[Patrick M. Hausen]

The other thing was when I finally got to get some firewalling out of it I had to add something similar to "Default allow LAN to any rule" for any interface to work properly, but this can't be right as it seems to allow inbound traffic  back to the interface.  I would normally expect to configure the outbound rule and the fw will know to return associated packets without having an explicit inbound rule ...

Rules are applied according to how the firewall sees the packets.

If a client system on LAN initiates an outbound connection to a host on the Internet or on the other side of a VPN tunnel, the first SYN packet from that client is coming IN through the LAN interface.

So you need a rule "LAN - IN - allow" for that particular traffic. You are right about the automatic state tracking and the return packets, no additional rules are necessary. In general apart from very special cases you do not need "OUT" rules.

[bickns]

Thanks Patrick - then it would seem my understanding is incorrect as I read that as "allow traffic IN to the LAN from  * "  - which I would normally read as "any interfaces".    However, when I read this again is see the default rules is to "allow traffic IN to the LAN from LAN net" .. doh!

It seems my actual problem is that I don't know *yet* where the intra interface routing is controlled - currently its happily allowing packets from a guest interface (the wizard gave it an opt prefix) to access the computers on the LAN (including the main menu) , which is clearly undesirable.  There doesn't seem to be anything in the auto generated options which enables this.    So scratching my head a bit in case its something like some special rule being applied from the settings.

[Patrick M. Hausen]

It seems my actual problem is that I don't know *yet* where the intra interface routing is controlled - currently its happily allowing packets from a guest interface (the wizard gave it an opt prefix) to access the computers on the LAN (including the main menu) , which is clearly undesirable.  There doesn't seem to be anything in the auto generated options which enables this.    So scratching my head a bit in case its something like some special rule being applied from the settings.

Please show the rule for your OPT1 interface.

[bickns]

Thanks - I even tried putting a block in at the end to block outbound on anything else and no joy ...   

    Protocol   Source   Port   Destination   Port   Gateway   Schedule      Description       
Automatically generated rules   
     IPv4+6 *   *   *   *   *   *   *   *   Default deny / state violation rule   
     IPv6 IPV6-ICMP   *   *   *   *   *   *   *   IPv6 RFC4890 requirements (ICMP)   
     IPv6 IPV6-ICMP   (self)   *   fe80::/10, ff02::/16   *   *   *   *   IPv6 RFC4890 requirements (ICMP)   
     IPv6 IPV6-ICMP   fe80::/10   *   fe80::/10, ff02::/16   *   *   *   *   IPv6 RFC4890 requirements (ICMP)   
     IPv6 IPV6-ICMP   ff02::/16   *   fe80::/10   *   *   *   *   IPv6 RFC4890 requirements (ICMP)   
     IPv6 IPV6-ICMP   ::   *   ff02::/16   *   *   *   *   IPv6 RFC4890 requirements (ICMP)   
     IPv4+6 TCP/UDP   *   0   *   *   *   *   *   block all targeting port 0   
     IPv4+6 TCP/UDP   *   *   *   0   *   *   *   block all targeting port 0   
     IPv6 CARP   *   *   ff02::12   *   *   *   *   CARP defaults   
     IPv4 CARP   *   *   224.0.0.18   *   *   *   *   CARP defaults   
     IPv4+6 TCP   <sshlockout>   *   (self)   22 (SSH)   *   *   *   sshlockout   
     IPv4+6 TCP   <sshlockout>   *   (self)   443 (HTTPS)   *   *   *   sshlockout   
     IPv4+6 *   <virusprot>   *   *   *   *   *   *   virusprot overload table   
     IPv4 UDP   *   68   255.255.255.255   67   *   *      allow access to DHCP server   
     IPv4+6 UDP   *   68   (self)   67   *   *      allow access to DHCP server   
     IPv4+6 UDP   (self)   67   *   68   *   *      allow access to DHCP server   
     IPv4+6 *   *   *   *   *   *   *   *   let out anything from firewall host itself   
     IPv4+6 *   (igb0)   *   ! WAN net   *   WAN_GW   *   *   let out anything from firewall host itself (force gw)   
        IPv4 *   guest net   *   *   *   *   *      Default allow guest to any rule      
        IPv4 *   *   *   WAN net   *   *   *      guest to WAN

[Patrick M. Hausen]

Code: [Select]

IPv4 *   guest net   *   *   *   *   *      Default allow guest to any rule     
IPv4 *   *   *   WAN net   *   *   *      guest to WAN
The first of these two rules allows systems on the guest/OPT1 interface with a source address of "guest net" to communicate with everything ("*") - including systems on LAN, of course.

The second rule does not make sense - you can delete it.

"WAN net" is not "the Internet". It is the local network directly connected to your WAN interface.

To achieve Internet but no LAN access for your guest net, create two rules in this order:

from: guest net
to: LAN net
action: deny

from: guest net
to: *
action: allow

The second already exists.

[bickns]

Thank you.  Now I get it ... I think.  My second rule was an "out" rule - as WAN is the ultimate destination, but the next destination is firewall not WAN - so this rule won't get evaluated ?   Probably explains the log entry having a rule "let out anything from firewall host itself"

It looks like I need to check all my rules so they are "in" rules and then go from there.  I've been searching the internet all day to see where the policy rules are set but its not obvious from the GUI - been a few years since I had to use iptables rules as I've been using watchguard so forgot to think about how the chains work.