Horrible latency / packet loss when using PPPoE over vlans

[stockerw]

Hi Folks,

I have OPNSense running on a i5-1240P (12 cores, 16 threads), 16GB RAM w/ 2x Intel 10GbE (82599ES) + 4xI-226V.

I have a weird issue that's been occurring since I got this device, that has never occurred before.

I have CityFibre 900Mbps fibre coming in to an ONT that presents on an ONT Port using vlan 911.

If I configure the WAN to use PPPoE with PPPoE configured to use vlan.911 on either a port of the i-226V or one of the 10G nics I get horrible packetloss and latency.

If I plug the ONT into my switch, strip the vlan and connect it to a port on the v226v the issue vanishes.

I've been pulling my hair out, I've tried MTU/MSS changes but just can't get to the bottom of the issue, and really don't want to be burning two switch ports just to strip this tag. (running short and not keen on an upgrade and this seems like a silly band aid.)

Really looking for tips at what else to look at and why this could be the case. It seems PPPoE on this equipment hates the vlan tag, where as my previous much lower power equipment had no issue.

TIA

stockerw

[meyergru]

It seems very likely that the VLAN on top of PPPoE reduces your MTU so much as to cause problems. You should first try to enlarge your MTU to be able to fit PPPoE and LAN overhead.

See [img width=https://forum.opnsense.org/index.php?msg=220936]http://this thread[/img] on how to do this - provided that your hardware and your ISP can do it.

You can also check what maximum MTU you have with this.

[stockerw]

Yeah, unfortunately I have already tried setting my MTU on my WAN interface to 1488 to allow the additional 4 bytes, with no success.

FWIW the 10G adapter is on jumbo to the switch as well.

[meyergru]

That is not exactly what I suggested. Yes, you can also lower the MTU, I always advocate to accomodate the usual 1500 bytes by enlarging the underlying interface's MTU.

Also, did you actually verify what exactly goes thru?

[stockerw]

I tried both increasing the MTU, to 1512 on the vlan parent interface, setting the wan to 1508 (PPP interface then = 1500).

AND

lowering the MTU on the WAN iface to 1492 & 1488.

Have confirmed the providers ONT supports baby jumbos, so we should in theory be fine, but alas still running into horrible latency, worse with 1512 than with 1492&1488 however.

Double checked the switch and no MTU mismatch/weirdness there.

Haven't looked at the script, but wasn't sure if you can even run bash from an opnsense shell but did reboot each time and in the shell

Cheers,

W