wireguard default MTU setting changed logic

[alfrisch]

Hi all,

in our setup we use jumbo frames through a wireguard VPN tunnel between two OPNsense instances together with OSPF. After the upgrade to 24.1 the dynamic routing stopped working and I could find in the logs

[XZZ9Y-NNTMQ][EC 134217741] Packet[DD]: Neighbor x.x.x.x MTU 8920 is larger than [wg2:x.x.x.x]'s MTU 1420

After switching temporarily to static routing and some hours of debugging I was able to trace down the problem to the MTU logic used in wireguard. In the pre 24.1 OPNsense instance it says in the help for MTU option

Set the interface MTU for this interface. Leaving empty uses the MTU from main interface which is fine for most setups.

But in the 24.1 instance it simply says

Set a specific device MTU for this instance.

Thus, the MTU from the parent device, which is set to 9000, seems not to be considered anymore with 24.1 and OSPF refused to work!

Is this change in the default logic in wireguard intended?

This was a nasty bug, because the MTU option is also hidden in "advanced mode", but setting MUT to 8920 made everything work again right away!

Cheers,
Albert

[franco]

Hi Albert,

What you describe sounds like it should be. I'm in doubt of "uses the MTU from main interface" which means what exactly? What's the main interface of a wireguard instance? There has never been a reference to a physical device as far as I know so nothing can ever use the same MTU from somewhere else and a network device always ends up with a default MTU even if we don't set it.

https://github.com/opnsense/core/blob/dca47d1c7fcf5a79837b84057fd1c54a70053a02/src/opnsense/scripts/Wireguard/wg-service-control.php#L80-L82

The code indicates we don't touch the MTU if not set. It uses "something" but the definition of this is fluent.. probably the default MTU hardcoded in the kernel.

One note is that if you use an interface assigned to your instance that MTU setting over there will overwrite the default MTU even if you didn't specify it in the WireGuard instance.

Did you use wireguard-go or wireguard-kmod before?


Cheers,
Franco

[alfrisch]

Hi Franco,

I used the kmod version before.

I do have an interface assigned to the wireguard tunnel, see attached screenshot, with the manual MTU setting present.

But when I keep the MTU option field blank, I get the "wrong" MTU of 1420.

Are you suggesting that the wg setting is supposed to be overwritten by the setting in "WAN0site2" and thus we should define it there manually? Because if I try so I get the error message

Cannot assign an IP configuration type to a tunnel interface.

minor note: because I installed several upgrades in a bunch, I noticed now that this behavior is not new in version 24.1 but already appeared when upgrading from 23.7.5 and 23.7.12_5

cheers, Albert

[marcquark]

You can specify the MTU that the created interface is supposed to get in the wireguard "Instance" settings, but you have to toggle "advanced settings" for the field to appear

[alfrisch]

Thanks for clarification, this is also as I understand and configured our system at the moment.

So to close the discussion in this thread, my take home message from Franco is that the MTU configuration logic change for  wireguard is indeed as intended. Thanks a lot!