Can't open some websites with https. MSS clamping on VLAN helps. Why?

[schossel]

Hi,
I have a weird issue since 2-3 days. In the beginning I wanted to renew some certificates with certbot running in a VM in it's own VLAN. I couldn't get new certificates and began to research.
From this VLAN10, I can ping kasapi.kasserver.com but I can't open https://kasapi.kasserver.com/dokumentation/ in a browser nor can I curl it. It doesn't work on any machine in VLAN10.
If I use a VPN or any other machine in my network, not in VLAN10, it works flawlessly. It worked in VLAN10 until a few days ago, I got certificates etc.
Because I had some similar issues some time ago, with a wireguard tunnel where I couldn't open some websites,and it was MSS in the end, I changed the MTU of VLAN10 in increments down but with no luck. Then I changed MSS of VLAN10 down and with 1472 it starts that I can curl https://kasapi.kasserver.com/dokumentation/ or open it in a browser. LAN has no problems. Has anybody an idea if this causes any trouble with MSS 1472 on that interface and why is that? This worked without problems for about 1-2 years now!?

This is the interface:

[Patrick M. Hausen]

VLANs do add 4 bytes of 802.1q header to each datagram. That does not match your observed number very well, though. What is the MTU on the parent interface of VLAN10 (on OPNsense) an what does the rest of your infrastructure look like (trunk ports, switches, ...)?

[schossel]

The parent interface is LAN which has no special MTU set, so I think it's 1500?
The only odd setup here is the double NAT with a FritzBox in front of the Opnsense, I could imagine that it adds some bytes but it was never an issue.
I'm searching for 2 days now and while I'm typing here, it's no more possible to open https://kasapi.kasserver.com/dokumentation/ from a LAN device. I just edited the LAN interface to MSS 1472 and now it's working again!
I did the same test with curl directly behind the FritzBox and before teh Opnsense and it works. So it could be some double NAT thing?!
Perhaps this issue exists for longer than I thought on VLAN10 but it was never an issue because it seems to work sometimes and sometimes it doesn't. There's nothing special with that website except that lego tries to connect to it to get new certificates from all-inkl.
I tried a few other https which came to mind and I'm not able to replicate this specific problem except for that site.
I don't think Switch etc. play any role in this, now that it also occured on LAN. My PC is not in any VLAN and directly connected by a normal 8 port switch without management. This morning when I made my original post, everything was working on LAN. Can it be something special with kasserver.com?
Does it have any negative side effects running the interface with MSS 1472?

[meyergru]

The fact that this did not occur earlier does not mean it was not broken all the time.

MTU/MSS is essential for any connection for which PMTU discovery does not work. Whatever the reason for this specific site may be, there seem to be packets that get dropped because of their length.

There may be several factors that grow packet sizes underway, like VLAN tags, VPN or PPPoE encapsulation (or a combination of those). Potentially, there is a performance drawback with smaller packets, but with TCP, there is no wait for turnarounds, because there can be many unackowledged packets in flight. So in reality, the effect of a smaller MTU is neglegible, even more when it is 1472 byte instead of 1500.

[schossel]

Ok, I'll give it a try on the VLAN for a few days with MSS 1472 and will see if it has any negative side effects.
Lowering the MTU had no effect whatsoever, is that normal? MSS is the right switch to tweak it?

[schossel]

Ooooookkkkk,
I just saw that there were a few updates and I installed them and the Opnsense rebooted (which I did a few times the last 2 days) and now it works again on all interfaces without MSS clamping even on VLAN10, as it was 2-3 days ago.
Weird.