I've been setting the NIC MTU to 1508 and the pppoe setting was (on an earlier version of OPNsense) also being set to 1500. I was checking some other settings recently and I noticed that the pppoe interface was now only set to 1492.
This is for the current OPNsense 23.7.1_3-amd64 release:
igc0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc2: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc3: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
igc4: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
enc0: flags=0<> metric 0 mtu 1536
pfsync0: flags=0<> metric 0 mtu 1500
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
pppoe3: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
ovpnc1: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1500
Have I missed something or is this an error?
I'm unsure when this started (from the looks of it FreeBSD 13.2 due to OPNsense 23.7 but I haven't checked older versions because it wouldn't change much) yet I've done a number of improvements due to this that I'm not comfortable pushing to 23.7.x just yet.
First one going into 23.7.2 is https://github.com/opnsense/core/commit/04941c1ef1e5 which just aligns the MTU read from the PPPoE config vs. interface config.
Next steps are adding proper MTU to parent interfaces (if the MTU is too small and the parent's interface config will allow it) also taking into account VLANs and QinQs and then also writing the correct MTU to the pppoe0 device when assigned, which is currently wrong: the GUI states a calculated value but assigns the input value to the device, not the calculated one.
The commit situation is a bit messy, but the new development release (included in 23.7.2) will have all the necessary bits if you are willing to try it out.
Cheers,
Franco
Hi Franco
Thanks for your quick reply and yes, I'm always willing to test a new version. What do I need to do for updating to the next release version? :)
Hi Bill,
When you update to 23.7.2 in a day or a few and everything looks normal switch to "development" release type and check for updates again and install. Do a reboot and check resulting MTU in ifconfig. It should correspond to GUI values: calc. value on pppoeX and actual value on hwX.
Switch back to release type before going to 23.7.3. Hopefully done :)
Cheers,
Franco
Thanks Franco
I'll give that a spin and let you know what happens.
To avoid confusion: 23.7.2 is due tomorrow.
Cheers,
Franco
Hi Franco
I upgraded to the development release and the pppoe connection shows the correct MTU=1500, I've now switched back 'normal' and wait for the next update. Thanks again. :)
Quote from: phoenix on August 23, 2023, 04:38:14 PM
Hi Franco
I upgraded to the development release and the pppoe connection shows the correct MTU=1500, I've now switched back 'normal' and wait for the next update. Thanks again. :)
Good news for you: It has already been released.
He means 23.7.3.
Thanks for testing!
Cheers,
Franco
Hi Franco
I've just updated to the latest release and the MTU is not being set to 1500, I've made the change, applied it and rebooted and no change - I tried 1500 & 1508 but neither worked.
igc0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc2: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc3: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
igc4: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
enc0: flags=0<> metric 0 mtu 1536
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
pfsync0: flags=0<> metric 0 mtu 1500
pppoe3: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
ovpnc1: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
Is there anything else I can try?
Regards
Bill
Mine works: set 1508 on WAN and pppoe0 has 1500 after reboot... Do you have custom PPP settings in the device configuration (not the WAN configuration)... these are hidden under advanced button.
Cheers,
Franco
Quote from: franco on August 22, 2023, 04:00:50 PM
When you update to 23.7.2 in a day or a few and everything looks normal switch to "development" release type and check for updates again and install. Do a reboot and check resulting MTU in ifconfig. It should correspond to GUI values: calc. value on pppoeX and actual value on hwX.
I'm still a bit confused actually... Using RFC4638 Baby-Jumbo's succesfully for quite some time with OPNsense, even wrote a small howto some years ago https://forum.opnsense.org/index.php?topic=21207.msg99312#msg99312 .
Only by reading this forum I discovered that my pppoe0 interface also failed to set the correct MTU value in latest release(s).
Did the upgrade from 23.7.2 to 23.7.3 and now the MTU value switched from 1492 to 1508 on the pppoe0 interface where I expect to see the "calculated" value of 1500 like it used to be. Is this by design now ?
pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1508
description: WAN_1_FTTH (opt33)
You probably need to apply interface at least once or reboot?
> Is this by design now ?
The design of this was multiple designs stacked on top of each other and none were working correctly in the median cases.
Let me quote Office Space: "... we fixed the glitch."
I'm not sure if this even changed, but there were so many problems with it that multiple rounds of fixes have been/will be made:
23.7.2 uncloggs the calculation of MTU values from a parent (hardware) interface or when you give strict MTU values from the PPP device advanced configuration this value will be adhered to without subtracting the PPPoE header size... the problem here is the resulting MTU is merely in the respective mpd.conf file, but not on the interface.
23.7.3 fixes the pppoe device MTU according to how it changed in version 23.7.2 and avoids setting the wrong MTU later when this was supposed to go to the hardware parent all along (that's what you input in interface settings, but it was set wrongly on the pppoe device instead).
23.7.x will fix the parent MTU situation also for VLANs and VLAN stacks (QinQ) all the way back to the non-VLAN parent device while adhering to the MTU set for these other interfaces if that happens to be the case. This will also allow to add the appropriate headroom for the VLAN header in each step.
An example:
PPPoE WAN MTU interface set to 1508 (calculated 1500)
pppoe 1500
qinq01 1508
vlan01 1512
hw0 1516
The "calculated" hint is just that unfortunately. Also at play is that if a smaller MTU is forced on ANY of the parents the MTU will fail to apply. Furthermore I think the PPPoE MTU on the device it self (not the config) is not really being adhered to anyway and that's why this went unnoticed so long.
Factor in another MTU bug here... QinQ requires its parent to have an MTU + 4, but VLAN does not require to have a hardware MTU + 4. I forced the latter in the code now. At some point somebody in FreeBSD will find this one and fix it.
Long story short there is quite a lot of slop in OPNsense and FreeBSD regarding the MTU handling. It's been fun, but to me it seems solved (at least as far as the development version is concerned).
Cheers,
Franco
PS: As said before please also check the PPP device advanced settings. If you ever put an MTU there it will be used rigorously.
Quote from: franco on August 30, 2023, 09:45:22 PM
PS: As said before please also check the PPP device advanced settings. If you ever put an MTU there it will be used rigorously.
Apologies, so for some clarity for me...
For a 'straightforward' PPPoE WAN, with no VLANs or anything to get a working 1500MTU I had to set the MTU in the [WAN] Interface to 1508 (calculated 1500) and then assign the Parent/Hardware port (igb0) as a 'live' interface and name it ignore or something and set just a 1508 MTU on that interface (nothing else) to force the 1508 MTU on that Parent/Hardware port to allow a 1500 ppoe mtu + 8 header to 'fit'. That was irritating as it creates an unneeded entry everywhere for this 'interface' that doesn't do anything.
Are we saying I can now remove/unassign that superfluous interface and if I set the MTU in PPP advanced settings it will force/honour the correct MTU for the hardware interface?
> Are we saying I can now remove/unassign that superfluous interface and if I set the MTU in PPP advanced settings it will force/honour the correct MTU for the hardware interface?
The PPPoE device MTU is used verbatim now, yes. You put the one you want on the PPPoE connection and the default is 1492. Anything bigger is already "jumbo". But it will not adjust parent MTU required for jumbo scenarios.
That latter part is fixed in the development version and as stated above not yet shipped in 23.7.x, but feel free to try the development release type from the firmware settings... and in this case you can ignore the PPPoE advanced setting for MTU like before.
Cheers,
Franco
Got it, thanks.
Apologies I missed the 'expanding/fixing' parent interface MTU automatically/invisibly to accommodate the PPPoE MTU+Header was in a future release. I'll sit tight, but sounds good [emoji106]
(WAF will not allow me to play with development builds [emoji23])
Sent from my SM-S918B using Tapatalk
Ok no worries. We will get there :)
Hello all,
for me the problem remains, i have opnsense in a vm in proxmox with proxmox MTU 9000 on the HW interface and i use VIRTIO for opnsense.
I have a "WANRAW" parent interface set to 1512 MTU, the WAN interface with 1508 MTU and since version 23.7 when mtu started to be 1492, although now i have updated to 23.7.3, after reboot the MTU is 1492.
If i change any setting on the wan interface and press "save", then the MTU goes to 1500.
To avoid chasing shadows it would be good to confirm/deny on the development version. As I said I haven't pushed all patches and I don't know all side effects that mpd5/kernel are imposing.
I can only attest (and it was confirmed) that the code does set it correctly when invoked so something else is happening or the relevant log errors are not posted.
Cheers,
Franco
Hello franco,
I am NOT on developement version, normal community edition. As i mentioned, if i just press save without any change to any of the Wan interface or to the PPPoE device, the PPPoE connection gets renewed with MTU 1500. But if i reboot, it starts with 1492.
Which logs should i post?
Thank you
# opnsense-log | grep ifconfig
Cheers,
Franco
Hi Franco ,
thank you for the reply.
the command "opnsense-log | grep ifconfig" returns empty. no results.
this is what i get with the ifconfig command:
root@OPNsense:~ # ifconfig | grep mtu
vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet2: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet4: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1508
vtnet5: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
vtnet6: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
vtnet7: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet8: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
vtnet9: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet10: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet11: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet12: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet13: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet14: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
vtnet15: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 65520
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
enc0: flags=0<> metric 0 mtu 1536
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
pfsync0: flags=0<> metric 0 mtu 1500
pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
ovpnc1: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
ovpnc2: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
zt55cpnf57tvnd5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 5000 mtu 2800
wg1: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
wg3: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
I've made a backport of the patch for 23.7.3:
https://forum.opnsense.org/index.php?topic=21207.msg174409#msg174409
Is pppoe0 is on top of vtnet5? Maybe vtnet driver is fishy in this regard.
Cheers,
Franco
hi,
pppoe is on top of vtnet4.
thank you. will look at the backport patch.
------update-------
applied the patch and now the pppoe mtu after reboot is set to 1500 :D
The backport patch is from the development version or the pre 23.7 version? (Just curious)
thank you franco.