Routing Internet over IPSec tunnel, certain sites failing

[mdriesnj]

Hello all,

   I feel like I've exhausted my options on this matter and I'm looking for help from the community. This should be quite simple but is driving me bananas.

The setup:
Two sites, A and B, both running 23.1.11
Site A LAN vlan IP 192.168.1.0/24
Site B LAN vlan IP 192.168.2.0/24

I have an IPSec, route based VPN set up and everything is routing just fine, both sides can reach each other. I'm attempting to route certain host's Internet traffic from Site A through the VPN tunnel and out of the Site B's WAN interface. 

I've setup the necessary policy with the gateway selected. I've created the required NAT rule at Site B to NAT 192.168.1.0/24 to Site B's WAN interface. Everything SEEMS to be working properly. I can ping pretty much anything on the Internet but certain web sites fail to load.

For example I can ping www.yahoo.com but if I try to load the site in my browser, or do an openssl s_client connect, the connection never completes. I've pulled PCAPs and see the SYN,SYN/ACK, ACK handshake but then I see retransmits and the session eventually falls apart.

Oddly enough, I can reach sites like apple, microsoft, bing and google but youtube, yahoo and some other sites just time out. I can ping them all and even trace route (ICMP or TCP) successfully.

I'm not using a proxy, Site A has BIND configured and Site B just unbound. Even If I switch the host to use quad 8, (in an attempt to simplify as much as possible) I get the same results.

Both sites had Juniper SRX firewalls up until recently when I replaced them both with Opnsense and new hardware. The same setup was working fine with the Juniper config. It's almost as though certain routers on the Internet aren't handling the traffic for some reason.

This has been a head scratcher that I just can't seem to figure out. Any help or insight is greatly appreciated.

[mdriesnj]

UPDATE:

Looks like the issue boiled down to MTU and MSS size. After doing some digging I found that duplicate ACKs are usually caused by MTU issues. So I hard coded my tunnel interface MTU to 1400 and MSS to 1360 and all of the sites which were broken are working. I have to do some more testing but I believe this solved it.