Fragmented UPD not send over IPSEC tunnel

[jaco.vandenberg]

Hi,

we are running IPSEC to a connect a site to a Fortigate firewall in the datacenter using a site-to-site VPN.
This works fine for normal  client traffic (mostly RDP over TCP) .
It turns out however, that Fragmented UPD is not send from the Fortigate site to the opnsense site .

This used to worked fine when the opnsense was a m0n0wall firewall, , however as soon as ik bring down the m0n0wall and lanuch the opnsense, it stops working. Switching back to the M0n0Wall it works fine again, so it muist be a opsense thing, blocking fragmented UDP for some reason.

opnsense is the latest version.

Any ideas what's wrong ?

[mimugmail]

Firewall / Settings / Normalization

Tick IP Do-Not-Fragment and recheck.

[jaco.vandenberg]

Excellent !

I tried that, your suggestion did not work at first sight, however, in this menu:

Firewall: Settings: Normalization : Detailed Settings, i added a line for the IPsec tunnel with the "IP Do-Not-Fragment" setting. That improved things quite a bit !

Now it allows most packets through the tunnel . However a certain packet loss is observed when tested with iPerf.

So upstream there is no UDP packet loss (there never was), downstream there is about 20% packet loss.
When the do-not-fragment setting ïs disabled, all packets are dropped, so it DOES have positive effect on the problem, however, quite some packets still get lost. 

That is strange, isn't it ?

[mimugmail]

Yep, but normally you should fix the fragmentation issue if possible, this would be best practice.

[bartjsmit]

Do you have the same MTU all along the path?

Bart...

[jaco.vandenberg]

did not check that, however TCP is flowing perfectly and the problem stays with very small MTU sizes (i.e. 400).

[mimugmail]

When the problem stays with small MTU too your hardware is underpowered.

I also experienced issues when testing with IPerf some ASA 5515 routers, when it reaches the limit packets get dropped.

[IJH]

Have you modified the MTU on the Fortigate policy?

you can modify it per policy:

config firewall policy
edit xxx
set tcp-mss-sender 1436

You'll need to calculate the MSS and edit the 1436 number depending on what you're connected by on the interface the Fortigate is using for the IPSec tunnel.

[mimugmail]

MSS is for TCP, the problem here are UDP packets