Messages

Reading this document (posted elsewhere on the forum) https://datatracker.ietf.org/doc/html/rfc8290 showed me a number of the common recommendations are just, well, wrong.

I ran into a lot of that as well when I was first figuring things out.

FQ-CoDel quantum should be set at your WAN MTU (in my case, 1514 bytes)

100% true, don't know why people would want to split packets up.

FQ-CoDel limit doesn't really need messed with, this setting defines the maximum number of packets that CAN be queued.  The default is 10240.  Most recommendations are to drop this value significantly, thus, causing the console flood messages.  For me, on my download pipe, I left this at the default of 10240.  For my upload pipe (with a max speed of 40Mbps) I halved it to 5120.  No real reason to, but my uploads will never saturate it.

Scaling the limit with bandwidth can help with Bufferbloat. I have ATT 500mbps symetric fiber, and the first hop is the converter box that I have bypassed. With a 2ms target and 4ms interval, 1200 limit is best and I never get any of the warnings. With a higher limit, it has a tendency to burst much higher for a longer interval which increases packet latency.

But yeah, the OP's limit is set too low if he's getting those warnings. It needs to be increased.

Recipe for ATT 500mbps (they over provision to about 640mbps):
Pipe: ATT-IN
Bandwidth: 560
Scheduler Type: fq_codel
Target: 2ms (should be much higher for many, ping your upstream gateway to get good numbers).
Interval: 4ms
Quantum: 1514 (Packet Size, see https://man.freebsd.org/cgi/man.cgi?ipfw)
Limit: 1200 (about perfect for 500mbps, 1500 is worse for me)
*only enable ECN on outgoing pipes/queues.

Pipe: ATT-OUT
Bandwidth: 560
Scheduler Type: fq_codel
Target: 2ms (should be much higher for many, ping your upstream gateway to get good numbers).
Interval: 4ms
Quantum: 1514 (Packet Size, see https://man.freebsd.org/cgi/man.cgi?ipfw)
Limit: 1200 (about perfect for 500mbps, 1500 is worse for me)
ECN: Enabled

Results: https://www.waveform.com/tools/bufferbloat?test-id=1ece2bf6-b7a2-4a40-9817-c90db7eafcd1
Best: 13ms+0+0
Any Day: 14ms+1+1/15ms+0+0

[if appropriate]

I have ATT 500mbps symmetric fiber.

Pipe: ATT-IN
Bandwidth: 540
Scheduler Type: fq_codel
Target: 2ms
Interval: 4ms
Quantum: 1514 (Packet Size, see https://man.freebsd.org/cgi/man.cgi?ipfw)
Limit: 1200 (about perfect for 500mbps)

Pipe: ATT-OUT
Bandwidth: 540
Scheduler Type: fq_codel
Target: 2ms
Interval: 4ms
Quantum: 1514 (Packet Size, see https://man.freebsd.org/cgi/man.cgi?ipfw)
Limit: 1200 (about perfect for 500mbps)
ECN: Enabled

---
Queues:
- HIGH-IN/OUT
- LOW-IN/OUT
Rules:
-IN & OUT rules for pushing ACK packets under 140 bytes to HIGH.
-IN & OUT rules for pushing DNS to High (by IP) to HIGH.
-IN & OUT rules for pushing regular traffic to LOW.

Notes:
-For rules I use the direction parameter to specify in/out instead of source/dest.
-I only do ECN for outgoing packets, because windows and linux now support it, but not every router/system between you and the internet host probably will.
-Quantum should be large enough to fit a packet, limit should be scaled with bandwidth.
-Don't be afraid to play with super low target/intervals with fiber if appropriate. It could be pretty low latency to your first hop. Do a ping to the first gateway and set your target/interval appropriately.

Results: https://www.waveform.com/tools/bufferbloat?test-id=1ece2bf6-b7a2-4a40-9817-c90db7eafcd1
Best: 13ms+0+0,
Any time every day: 14ms+1+1 / 15ms+0+0