Suggestion for Bufferbloat fix. Fibre to the Home. No PPoE.

[Seimus]

Linux: https://www.waveform.com/tools/bufferbloat?test-id=964b7180-4a1f-4eed-a114-1dfb613e9b63
Win10: https://www.waveform.com/tools/bufferbloat?test-id=edad2d94-d2c8-41e1-8b63-a31eeb2539bb

These results are not bad, and if they are constant I would call it a win.

But as mentioned by @meyergru you may have differences due to congestion algorithms. On Win10 I think the default is CTCP or CUBIC.

I personally run BBR on Linux.

Code Select Expand

bat /etc/sysctl.d/bbr.conf
─────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     │ File: /etc/sysctl.d/bbr.conf
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ net.core.default_qdisc=fq
   2 │ net.ipv4.tcp_congestion_control=bbr
─────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Regards,
S.

[OPNenthu]

It was worth a try guys, but I'm not seeing a difference between BBR and CUBIC on my system.  At least the loaded latency in all cases doesn't go above +10ms.

[RamSense]

I have been relentlessly trying all sort of guides and settings for bufferbloat. I have finally found a solution (1 gig fiber PPPOE), and getting A rates at waveform (https://www.waveform.com/tools/bufferbloat)

I will call this used guide guide "Dirty but effective"
That is because it states to use FQ-CoDel quantum of 3000.
But when I change the quantum to the described 1500 (mtu of connection) I get a B instead of A.

And in addition to this above guide, I added the [controle plane / ICMP ] as described on this forum elsewhere.

So although I do not understand why and it is not right or recommended to use 3000 quantum, I have to say it works great over here.

[Seimus]

I will call this used guide guide "Dirty but effective"

The creator of this article looks like doesn't know what their are doing.
There are several miss statements about Pipes and Queues.
There are several miss-configurations in the Pipe > BW, Queues, Quantum
There are several miss-configurations in the Queue > MASK, ECN, Enabled Codel

And so on.

Some of the configured features don't do anything and some of them have impact. Like MASK, Quantum & BW and overlapping FQ_C in scheduler and CoDel in the Queue, causes overall that there is twice running the CoDel algorithm, fighting each other.

In layman terms Quantum basically handles how much bytes at once can from a Flow queue leave (FQ_C internal one). The reason why you want to use Quantum at your MTU size when your BW is above 100Mbit is to serve 1 packet per flow (there can be thousands flows, 1 flow = 1 internal FQ_Codel queue). Higher Quantum sizes could starve smaller packets. And too small Quantum sizes could starve bigger packet sizes. Both related to the configured Interface MTU.


Regards,
S.

[cookiemonster]

Maybe that is due to the TCP congestion algorithms used. You can change it with Windows, I think under Win10, it was BBR2, but that had some problems, so they reverted back to CUBIC for Win11.

With Linux, you can easily change it via sysctl. These are the values I use:

Code Select Expand

net.core.rmem_default = 2048000
net.core.wmem_default = 2048000
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.ipv4.tcp_rmem = 4096 1024000 33554432
net.ipv4.tcp_wmem = 4096 1024000 33554432

# don't cache ssthresh from previous connection
#net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_adv_win_scale = 5
# recommended to increase this for 1000 BT or higher
net.core.netdev_max_backlog = 30000
# for 10 GigE, use this
# net.core.netdev_max_backlog = 30000
net.ipv4.tcp_syncookies = 1
# Enable BBR for Kernel >= 4.9
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr


Interesting. I did not know anything about this. Thanks @meyergry

Hey. I've been using a windows laptop for testing the bufferbloat so far. Normally I use linux but had a need to stay booted on Win last few days. This one is connected via a Wi-Fi 6 (802.11ax) Wifi network using a Intel(R) Wi-Fi 6E AX210 160MHz adapter. Depending on location I can get as little as 480/721 (Mbps) agregated link speed (rec/tran) so I have a bottleneck there at times. Wired connections are only one for a PC but I can't get to it most of the time.
For OPN's CPU I'm using an AMD Ryzen 5 5600U on Proxmox with two vCPUs. Just did a ubench run on it and gives: Ubench Single CPU:  910759 (0.41s). So I think that is Ok.
I've now reset the shaper to docs defaults. This time also the upload side. I need to reboot (had limit and flows on the pipe), I'll update the post.

HW should be okay to handle ZA + Shaper and that throughput.
But keep in mind the stuff about WiFi I mentioned above.


Regards,
S.

So far, gone back to exactly as docs I am getting consistent B grades. It seems to confirm my testing was flawed too. Wired testing seems better but don't have the values at hand.
That said, although I did know that I expected wired/wifi differences, I was hoping that the bufferbloat cure would help the wireless clients, which are the majority in the household, hence I was testing this way.
Is it possible or even desirable to tweak the shaper for wireless as main target ?