IPSec with aggregated networks dening local traffic

[pmladenov]

Hello,

I have an OpenSense Hub and Spoke topology with the following IP addressing schema:

Hub: 10.30.0.0/16
Spoke1: 10.31.0.0/16
Spoke2: 10.32.0.0/16
Spoke3: 10.33.0.0/16
...and so on
SpokeX: 10.X.0.0/16
SpokeY: 10.Y.0.0/16

Please note - all Spokes have a connection to the HUB only, there's no Spoke-to-Spoke physical direct link. The only way for SpokeX to communicate with SpokeY is via the HUB.
All Spokes and the HUB are OpnSense firewalls and there's no other firewalls/routes in the topology (only pure L2 switches and OpnSense FWs)

Of course, I need an encryption between all Spokes and the HUB (including Spoke-to-Spoke traffic).

What I did so far (which seems to work up certain extent) is:

Each Spoke has a static default-gw pointing to the HUB and the HUB has a static route for the corresponding Spoke's subnet pointing to the that Spoke opnsense FW.
Without encryption involved I have full connectivity between each spoke and HUB and between spokes itself (via the HUB)

Each Spoke forms an IPSec tunnel to the HUB and for phase 2 I have:

SpokeX:
Local Subnet: 10.X.0.0/16
Remote Subnet: 10.0.0.0/8

Hub (to Spoke X):
Local Subnet: 10.0.0.0/8
Remote Subnet: 10.X.0.0/16

That works fine for the connectivity between Spoke X and HUB sites, as well as Spoke X and Spoke Y (via the HUB site) but the problem I'm facing and looking for a solution is for local Spoke traffic:

For example - Spoke 1 OpnSense FW has 2 inside logical interfaces:

VLAN100 - 10.31.100.1/24
VLAN200 - 10.31.200.1/24

Local hosts (in Spoke1 site) living in VLAN100 (10.31.100.0/24) are NOT able to communicate with local hosts (again in Spoke1 site) living in VLAN200 (10.31.200.0/24). Even these hosts are NOT able to reach their default gateway. I assume their traffic is encrypted following 10/8 phase2 policy.

Is there any way I can exclude that local traffic from being encrypted? These 2 interfaces are locally connected to the same firewall, so apparently I would like to have clear text connectivity between both local subnets following OpnSense routing table (directly connected routes)

Can I have more than 1 subnet in phase 2 (local or remote)?

Any suggestions are highly appreciated!

Regards,
Plamen

 

[pmladenov]

After digging into that in the last 4-5 hours I've found the following solutions (which may save hours for others having the same problem)

Solution 1:

Go to route based IPSec (VTI)

Pros:
Good for more complex topologies.

Cons:
MTU and fragmentation issues (must disable interface scrub in order IP fragmentation to work properly, and I'm not sure what will happen with MSS in that case)
More complex (have to deal with tunnel IPs, new single gateways, complex routing, etc) 

My 2 cents:
VTI mode works really well on ...cisco equipment, but not so well here (at least not in my virtual lab with a little bit outdated software now). If FRR plugin had worked properly (in conjunction with HA setup with config synchronization and CARP) it would have been a perfect fit with VTI (route based) mode.



Solution 2:

There's an option to have multiple Phase 2 entries and specify multiple subnets (and not using 10/8 aggregate one)

Pros:
Simply works, no MTU issues, IP fragmentation works w/o any pf modifications.

Cons:
Simply works in small networks, but it will become real administrative pain if there're more spokes and you add additional ones over the time (you'll have to go to all other spokes and add the additional phase2 entry and also on the hub)

 
Solution 3 (the best fit for me):

VPN -> IPSec -> Advanced Settings -> Passthrough networks
I've added the local /16 here on each spoke, which simply tells not to encrypt any local traffic.

Pros:
Fastest and simplest one

Cons:
I can't think of any disadvantages at this point.


Hope that helps.

Regards,
Plamen