[Solved] OPNSense 20.7.2 broke 1508 MTU PPPoE

[oxydius]

Hi,

I just installed the 20.7.2 update and routing stopped working on my setup. I imported the config in 20.7 LiveCD mode and am up and running again.

After trial and error I realized my 1508 MTU on PPPoE was the culprit. I want to avoid the 1492 fragmentation and it had been working flawlessly but something must have changed in the last update. The fact that only Google would load while every other site would "wait for connection" or load partially was a good hint. 

Intel(R) Celeron(R) CPU 3865U @ 1.80GHz (2 cores)
igb0: <Intel(R) PRO/1000 PCI-Express Network Driver>
...
igb5: <Intel(R) PRO/1000 PCI-Express Network Driver>

My setup is WAN PPPoE (1508 MTU, 1500 PPP) over an ISP-specific VLAN, over a 2-port LACP LAGG connected to Gigabit Internet (fiber).

Did anyone else encounter this issue?

[hushcoden]

It still works for me...

[oxydius]

How is your WAN interface setup?

Setting 1500 as PPPoE MTU makes 20.7.2 work for me. With 1508 I can still ping up to 1472, but cannot load forum.opnsense.org.

[mimugmail]

1508 is too big as you have ppp overhead.
Did this really work with 20.7.1?

[oxydius]

Yes, 1508 PPPoE MTU worked with 20.7.1. After overhead it left 1500 bytes for WAN.

https://tools.ietf.org/html/rfc4638

I could try removing the LAGG from the equation as that also needs the MTU bump.

[staticznld]

No issues here, I had the problem problem before, when setting mtu on the pppoe interface on 1508.
I fixed it by creating another interface on the physical nic and set there the mtu to 1508 to.

https://www.speedguide.net/
Tcp analyzer shows here mtu 1500.

[hushcoden]

It definitely works for me, see attached ifconfig command.

[oxydius]

Thanks for the info guys. I still cannot get it working automatically but I can fix it manually at least. I removed LAGG to keep things simple.

Here's the standard sub-optimal setup :
WAN PPPoE 1500 - working
 igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 igb0_vlan35: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492

Here's my setup for the last 2 years :
WAN PPPoE 1508 - broken
 igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 igb0_vlan35: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1500

Manually fixed with "ifconfig igb0 mtu 1508" - working
 igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1508
 igb0_vlan35: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1508
 pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1500

It appears the PPPoE config doesn't pass MTU to underlying interfaces. Under the PPP configuration, "Link Parameters ( igb0_vlan35 ) MTU" has no effect. When set manually, igb0 must have its MTU increased to 1508 before igb0_vlan35 and it works as before, provided I don't reboot. staticznld, how did you go about creating another interface on the physical nic with 1508 MTU?

[staticznld]

Under interfaces choose your parent nic without vlan or pppoe and add it.
Enable the device and only set the Mtu to 1508.

[oxydius]

Thanks staticznld, I added a dummy interface to bump the MTU. That works.