Suggestion for Bufferbloat fix. Fibre to the Home. No PPoE.

[cookiemonster]

Hi. Another "help with bufferbloat".
I am currently on OPN 25.1.12-amd64 on a VM that has been running fine and has been updated a few major releases. All good.
At some point in the past, perhaps 2 years ago I followed one of the threads here to get decent bufferbloat help. It worked fine and I got B on waveform site with only the "low latency gaming measure" being !. That was good enough for me. I don't do gaming. I only need video (MS Teams / zoom ) to work reliably when needed.
My ISP package is fibre to the premise at 520 Down / 72 Up. Their speeds are normally consistent.

I had what seemed some buffering last week and went to check settings. I realised perhaps I needed to reconfigure it so I did a) read a few recent to a max 24 months posts; b) checked the current docs. I admit I can't understand the current way to use the "limit" note of the docs, the reference to the bug.

I decided to set it up per docs and made note of what I had first.
Result: consistently C results. Includes reboots when changing the flows.

I went back to what I had and still mostly C, sometimes B.

So this is the background. Can someone make a suggestion what values to use?
These are the values I had before the change but now the results _appear_ worse. And yes, it doesn't make a whole lot of sense but I'm looking for another set of eyes in case I've stared too long.

Download pipe

Code Select Expand

Enabled X
Bandwidth 490
Bandwidth Metric Mbit/s
Queue
Mask (none)
Buckets
Scheduler type FlowQueue-CoDel
Enable CoDel
(FQ-)CoDel target
(FQ-)CoDel interval
(FQ-)CoDel ECN X
FQ-CoDel quantum
FQ-CoDel limit 20480
FQ-CoDel flows 8192
Enable PIE
Delay 1
Description Download pipe
 
Download queue

Code Select Expand

Enabled X
Pipe Download pipe
Weight 100
mask destination
Buckets
Enable CoDel
(FQ-)CoDel target
(FQ-)CoDel interval
(FQ-)CoDel ECN X
Enable PIE
Description Download queue

Download rule

Code Select Expand

Enabled X
Sequence 1
Interface WAN
Interface 2 None
Protocol ip
Max Packet Length
Source any
Invert source
Src-port any
Destination any
Invert destination
Dst-port any
DSCP Nothing selected
Direction in
Target Download queue
Description Download rule 

[meyergru]

The mask in the Download queue should be (none). Also, you should define the Upstream side of things as well.

[pfry]

Is a downstream shaper (particularly a single queue) likely to have the effect you want? I used downstream shapers in the past, but my purpose was to control offered load by adding latency, using multiple queues on a CBQ shaper. I didn't bother after my link passed 10Mb; it did help at 6-10Mb.

I'd think a simple fair queue with no shaper would be the best option for you. I don't know the best way to accomplish that - perhaps open the pipe beyond 520Mb/s (toward single-station LAN speed). I haven't looked at the fq-codel implementation in... a while. The one I recall used a flow hash, and you could set the number of bits (up to 16, I believe). It looks like the ipfw implementation has that limit (65536). I'd think more can't hurt - fewer (potential) collisions. I wouldn't expect any negatives, but you never can tell. PIE just sounds like a RED implementation - I can't see that it'd have much if any effect, as I wouldn't expect your queue depths/times to reach discard levels.

Of course, you could have upstream issues, at any point in the path.

[cookiemonster]

The mask in the Download queue should be (none). Also, you should define the Upstream side of things as well.

yes I tried with that removed as per docs. Still bad.
Anything else you can spot?
Edit: p.s. uploads seem very good in the bufferbloat tests but I can add them to the thread no problem. I wanted to keep it as tidy as possible.

[cookiemonster]

Is a downstream shaper (particularly a single queue) likely to have the effect you want? I used downstream shapers in the past, but my purpose was to control offered load by adding latency, using multiple queues on a CBQ shaper. I didn't bother after my link passed 10Mb; it did help at 6-10Mb.

I'd think a simple fair queue with no shaper would be the best option for you. I don't know the best way to accomplish that - perhaps open the pipe beyond 520Mb/s (toward single-station LAN speed). I haven't looked at the fq-codel implementation in... a while. The one I recall used a flow hash, and you could set the number of bits (up to 16, I believe). It looks like the ipfw implementation has that limit (65536). I'd think more can't hurt - fewer (potential) collisions. I wouldn't expect any negatives, but you never can tell. PIE just sounds like a RED implementation - I can't see that it'd have much if any effect, as I wouldn't expect your queue depths/times to reach discard levels.

Of course, you could have upstream issues, at any point in the path.

You mean set it up as per the docs https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html ?
But I can try see if I follow the thinking and put a pipe beyond the 520 Mbps, to see what happens. Thanks for the idea.
Going a little mad with this at the moment.

Thing is, I have a decent (for me) 520 Mbps bandwith. Normally I wouldn't bother with shaping but I seem to have the odd buffering now after this change I made. Frustratingly it is not better ie back to normal after restoring the previous settings.

[cookiemonster]

To make it factual, my just-made 2 test results:
BUFFERBLOAT GRADE
B

LATENCY
Unloaded 26 ms
Download Active +39 ms

Upload Active +0 ms
SPEED ↓ Download 259.5 Mbps
↑ Upload 66.9 Mbps

Second:
BUFFERBLOAT GRADE
B

Your latency increased moderately under load.

LATENCY
Unloaded 21 ms
Download Active +42 ms
Upload Active +0 ms
SPEED ↓ Download 262.4 Mbps
↑ Upload 66.8 Mbps
==
So it's giving me Bs at the moment. Is this "good enough" leave-it-alone result? Tomorrow it might give me Cs though. I'll keep checking.