Shaping causes dmesg to throw fq_codel_enqueue maxidx = 801 over limit

[bachmarc]

Hello,
where ever I search I get only a "click by click" documentation of shaping that shows how to do it for a monkey but is not explaining the why and "what if"

the kernel tells with dmesg:
fq_codel_enqueue maxidx = 801
fq_codel_enqueue over limit

Ok, and now the clicky docu is at an end
I cannot find any meaningful docu between level "the algorithm - scientific papers" and "click here and do not ask, monkey"...

Crazy things like "do not check: Enable CoDel active queue management" are totally unclear to me.
googling my warning finds nothing but guessing

I beg for a nice link to a good docu

Marc   

[bachmarc]

It seems nobody really knows how to configure it right.. we all replicate the same steps like apes?

[fabianodelg]

Hi there,

I've spent many hours (days, months!) on the subject and now I'm fairly self sufficient and I quite understand how the shaper works. I'm happy to help if I can.

Let's start from the basics... what are you trying to achieve? What's your current configuration?

Best.

[bachmarc]

In the past on a linux-based firewall I had a more complex setup that was shielding my job (videocalls) against all the kids in the house that create traffic/bursts.

In Opnsense I thought to mirrow this but I found that the Codel stuff could do more than just keeping a fast track for my work. So initially I just shaped the traffic to avoid bufferbloat and postponed the further ideas.  It seems to work somehow, because I get an A rating in the "waveform buffbloat test" with the shaper and a B without.

I have a typical German asymmetric connection 500mbit down/ 50 mbit up.
There are things that make me suspicious about the shaper and as I cannot find a real documentation all ends with guessing.

a) I loss a significant part of my 500/50 with a switched on shaper. It differs but 430/42 are good days.
That feeds the idea that even a stupid throttle would reduce presure on router and deliver such stable ping times.

b) The kernel throws this over limit messages and I cannot get a context.


My setup is fairly simple:
2 Pipes
- 500mbit for down
  method: FlowQueue-CoDel, (FQ-)CoDel ECN enabled, quantum 1600, limit 1600
-50 for upstream
  method: FlowQueue-CoDel, (FQ-)CoDel ECN enabled, quantum 160, limit 160

3 Queues all (FQ-)CoDel ECN true
Down Prio 1
Up prio10
up prio 1

4 Rules
ACK, DNS i push through the prio Queue
all the rest stays in default queues

I followed a basic tutorial and there are things that confuse me but do not get explained, like why nobody enables CoDel active queue management. Seems to be not the obvious or the UI has a terrible naming convention.

If you could bring some light in this, I am happy

Marc




 

[apunkt]

I observe the very same issues as you did.

What was - and always is - helping for me was trying to understand what I do and why. The basic understanding for fq_codel parameters I found in this document https://datatracker.ietf.org/doc/html/rfc8290

I understand that in FreeBSD the implementation could be different, however it gives you a necessary, basic understanding.
Thus I believe the quantum recommendation of 300 / 100Mbps you read almost everywhere originates to quantum and target parameters in accordance with the VoIP packet size for low latency requirement that fits perfectly into the 300 as shown here https://www.ccexpert.us/unified-callmanager/voip-packet-size-and-packet-rate-examples.html

Before that I was pinning up what I found in several tutorials/howtos, which gave me nice benchmark/bufferbloat results, but terribly failed under real load.
I was able to find my own settings finally working.

Happy firewalling
a.

[Lakkiada]

The fq_codel length error message originates if the queue is full past it's limit. After reading extensively about codel configuration and all the conflicting information available, I've determined on my Asymmetrical coax connection (600/20) that the queue length ONLY needed to be lengthened on the Download queue to stop the messages spamming the logs (since the upload max is so much lower, codel may be tuned differently).

I have tested and retested many many times using:
https://www.waveform.com/tools/bufferbloat
https://fast.com/
https://www.speedtest.net/
https://speed.cloudflare.com/

I always receive an A+ rating with fq_codel and shaping ON. Without, anywhere from B to D pending congestion and activity.

All configurations to fq_codel take place on the pipes. Same as you witnessed- I have to severely lower my Down speed.

Download = 360Mb (can fluctuate, but X<400 most stable), Queue = 2, Type = fq_codel, Target = 12, Interval = 100, ECN = ON, Quantum = 300, Limit = 4096, Flows = 8192

Upload = 18Mb, Queue = 2, Type = fq_codel, Target = 12, Interval = 80, ECN = OFF, Quantum = 300, Limit = 1024, Flows = 8192

I use weighted queues to prioritize my desktop over other hosts, and rules to prioritize DNS and TCP ACKs into those weighted queues.

Hope that helps! Happy Computing!

Further resources:
https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/
https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html
https://www.lullabot.com/articles/eliminating-robots-and-voip-glitches-with-active-queue-management
https://isc.sans.edu/diary/Securing+and+Optimizing+Networks+Using+pfSense+Traffic+Shaper+Limiters+to+Combat+Bufferbloat/27102
https://github.com/opnsense/docs/blob/master/source/manual/how-tos/shaper_bufferbloat.rst

EDIT: Be advised- these settings are EXTREMELY specific to each use case (router hardware, network conditions/configuration, etc) and may not yield the same results.