PPPoE WAN (IGB0) max speed 250 MBits APU4D4

[Hopmeister]

Hello all,

I was using my apu4d4 (4 GB DRAM, 4 i211AT NICs) with OPNsense as my standalone fw/router. I managed to get an FTTP bandwidth / speed increase from 150/30 to 500/73 for no extra cost.

I spent an hour castigating my ISP for only giving me 250MBits and they told me that the line is reporting 510MBits.
So I spent a long day doing tests.

The APU4 NICS will do 1GB/s in IP mode, but will only do 250 (ish) MBits in PPPoE mode. Traffic shaping does not affect the speed only the bufferbloat score. I can get A+, A+, A+ but only at 250MBits. It will not go faster. When I use the ISP's router (IP not PPPoE) I can get 500MBits A+, A+, A+.

The final result is that in LAN / IP mode the NIC will run at 1 GB/s (near as) when it's connected to the ISP's router LAN port and when I use my other apu2e0 (2 GB DRAM, 2 i211AT NICs) with OpenWRT.

I finally found one post that stated that BSD has PPPoE issues and Linux (OpenWRT) does not. Is that true?

So I'm guessing that OPNsense does have a PPPoE issue, although it may be my hardware, but nothing I did would get the speed over 250MBits in PPPoE mode. I'm open to suggestions if it's possible as I'm trying to minimise boxes around the house, not increase them.

Thanks  ;D

[thewolf56]

I hope you get a helpful response.  The speed limitation is the only reason I have kept my ISP's gateway box in the chain as a transparent bridge in order to keep my 1Gbps connection, but lately, it has needed a reset every once in a while as it will drop to 100Mbps.

[franco]

There already is a number of threads with tunable tweaks about this including one about RSS that could help. I would encourage you to look for them.


Cheers,
Franco

[Hopmeister]

Thank you Franco, I'll have a look. I did try one tuneable but it made it worse. I'll search and if I get any improvement I'll update my thread  8)

[franco]

Ok, thanks. For RSS I meant this one: https://forum.opnsense.org/index.php?topic=24409.0


Cheers,
Franco

[Hopmeister]

Thanks Franco, I'm reading that now. I still don't see how it's a kernel or tuneable if IP can do 1GB why can't it do the same under PPPoE?

It is cutting 50% of the speed/bandwidth.

:D

[franco]

Because PPPoE is effectively single-threaded due to added PPPoE header and no appropriate mitigation for it exists in FreeBSD.


Cheers,
Franco

[bunchofreeds]

Please read this entire comment #11
It's unfortunate but true.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203856#c11

As PPPoE seems to become more popular for gigabit plus residential internet plans, hopefully this will be answered. But it requires an effort from hardware right through to OPNsense.