pppoe MTU not being set correctly.

[phoenix]

I've been setting the NIC MTU to 1508 and the pppoe setting was (on an earlier version of OPNsense) also being set to 1500. I was checking some other settings recently and I noticed that the pppoe interface was now only set to 1492.

This is for the current OPNsense 23.7.1_3-amd64 release:

Code Select Expand

igc0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc2: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc3: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc4: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
enc0: flags=0<> metric 0 mtu 1536
pfsync0: flags=0<> metric 0 mtu 1500
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
pppoe3: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
ovpnc1: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1500

Have I missed something or is this an error?

[franco]

I'm unsure when this started (from the looks of it FreeBSD 13.2 due to OPNsense 23.7 but I haven't checked older versions because it wouldn't change much) yet I've done a number of improvements due to this that I'm not comfortable pushing to 23.7.x just yet.

First one going into 23.7.2 is https://github.com/opnsense/core/commit/04941c1ef1e5 which just aligns the MTU read from the PPPoE config vs. interface config.

Next steps are adding proper MTU to parent interfaces (if the MTU is too small and the parent's interface config will allow it) also taking into account VLANs and QinQs and then also writing the correct MTU to the pppoe0 device when assigned, which is currently wrong: the GUI states a calculated value but assigns the input value to the device, not the calculated one.

The commit situation is a bit messy, but the new development release (included in 23.7.2) will have all the necessary bits if you are willing to try it out.


Cheers,
Franco

[phoenix]

Hi Franco

Thanks for your quick reply and yes, I'm always willing to test a new version. What do I need to do for updating to the next release version? :)

[franco]

Hi Bill,

When you update to 23.7.2 in a day or a few and everything looks normal switch to "development" release type and check for updates again and install. Do a reboot and check resulting MTU in ifconfig. It should correspond to GUI values: calc. value on pppoeX and actual value on hwX.

Switch back to release type before going to 23.7.3. Hopefully done :)


Cheers,
Franco

[phoenix]

Thanks Franco

I'll give that a spin and let you know what happens.

[franco]

To avoid confusion: 23.7.2 is due tomorrow.


Cheers,
Franco

[phoenix]

Hi Franco

I upgraded to the development release and the pppoe connection shows the correct MTU=1500, I've now switched back 'normal' and wait for the next update. Thanks again. :)

[Vexz]

Hi Franco

I upgraded to the development release and the pppoe connection shows the correct MTU=1500, I've now switched back 'normal' and wait for the next update. Thanks again. :)

Good news for you: It has already been released.

[franco]

He means 23.7.3.

Thanks for testing!


Cheers,
Franco

[phoenix]

Hi Franco

I've just updated to the latest release and the MTU is not being set to 1500, I've made the change, applied it and rebooted and no change - I tried 1500 & 1508 but neither worked.

Code Select Expand

igc0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc2: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc3: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc4: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
enc0: flags=0<> metric 0 mtu 1536
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
pfsync0: flags=0<> metric 0 mtu 1500
pppoe3: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
ovpnc1: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420

Is there anything else I can try?

Regards

Bill

[franco]

Mine works: set 1508 on WAN and pppoe0 has 1500 after reboot... Do you have custom PPP settings in the device configuration (not the WAN configuration)... these are hidden under advanced button.


Cheers,
Franco

[netnut]

When you update to 23.7.2 in a day or a few and everything looks normal switch to "development" release type and check for updates again and install. Do a reboot and check resulting MTU in ifconfig. It should correspond to GUI values: calc. value on pppoeX and actual value on hwX.

I'm still a bit confused actually... Using RFC4638 Baby-Jumbo's succesfully for quite some time with OPNsense, even wrote a small howto some years ago https://forum.opnsense.org/index.php?topic=21207.msg99312#msg99312 .
Only by reading this forum I discovered that my pppoe0 interface also failed to set the correct MTU value in latest release(s).

Did the upgrade from 23.7.2 to 23.7.3 and now the MTU value switched from 1492 to 1508 on the pppoe0 interface where I expect to see the "calculated" value of 1500 like it used to be. Is this by design now ?

Code Select Expand

pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1508
description: WAN_1_FTTH (opt33)


[franco]

You probably need to apply interface at least once or reboot?

> Is this by design now ?

The design of this was multiple designs stacked on top of each other and none were working correctly in the median cases.

Let me quote Office Space: "... we fixed the glitch."

I'm not sure if this even changed, but there were so many problems with it that multiple rounds of fixes have been/will be made:

23.7.2 uncloggs the calculation of MTU values from a parent (hardware) interface or when you give strict MTU values from the PPP device advanced configuration this value will be adhered to without subtracting the PPPoE header size... the problem here is the resulting MTU is merely in the respective mpd.conf file, but not on the interface.

23.7.3 fixes the pppoe device MTU according to how it changed in version 23.7.2 and avoids setting the wrong MTU later when this was supposed to go to the hardware parent all along (that's what you input in interface settings, but it was set wrongly on the pppoe device instead).

23.7.x will fix the parent MTU situation also for VLANs and VLAN stacks (QinQ) all the way back to the non-VLAN parent device while adhering to the MTU set for these other interfaces if that happens to be the case. This will also allow to add the appropriate headroom for the VLAN header in each step.

An example:

PPPoE WAN MTU interface set to 1508 (calculated 1500)
pppoe 1500
qinq01 1508
vlan01 1512
hw0 1516

The "calculated" hint is just that unfortunately. Also at play is that if a smaller MTU is forced on ANY of the parents the MTU will fail to apply. Furthermore I think the PPPoE MTU on the device it self (not the config) is not really being adhered to anyway and that's why this went unnoticed so long.

Factor in another MTU bug here... QinQ requires its parent to have an MTU + 4, but VLAN does not require to have a hardware MTU + 4. I forced the latter in the code now. At some point somebody in FreeBSD will find this one and fix it.

Long story short there is quite a lot of slop in OPNsense and FreeBSD regarding the MTU handling. It's been fun, but to me it seems solved (at least as far as the development version is concerned).


Cheers,
Franco

[franco]

PS: As said before please also check the PPP device advanced settings. If you ever put an MTU there it will be used rigorously.

[Taunt9930]

PS: As said before please also check the PPP device advanced settings. If you ever put an MTU there it will be used rigorously.

Apologies, so for some clarity for me...

For a 'straightforward' PPPoE WAN, with no VLANs or anything to get a working 1500MTU I had to set the MTU in the [WAN] Interface to 1508 (calculated 1500) and then assign the Parent/Hardware port (igb0) as a 'live' interface and name it ignore or something and set just a 1508 MTU on that interface (nothing else) to force the 1508 MTU on that Parent/Hardware port to allow a 1500 ppoe mtu + 8 header to 'fit'. That was irritating as it creates an unneeded entry everywhere for this 'interface' that doesn't do anything.

Are we saying I can now remove/unassign that superfluous interface and if I set the MTU in PPP advanced settings it will force/honour the correct MTU for the hardware interface?