Traffic shaping makes bufferbloat score worse

[beneix]

Ï used to have an XDSL connection that meant I had to use QoS to ensure my videoconferencing worked OK. When we were moved to fibre I just updated the bandwidth and quantum numbers in the traffic shaper settings and thought nothing more about it. Because of the much faster connection, things mostly work OK.

Then I decided to run a bufferbloat test and was surprised to see I only got a "C" rating from https://www.waveform.com/tools/bufferbloat. I connected via Ethernet directly to the OPNSense router and ran the test again - same result. Then I tried turning traffic shaping off and now I got a rating of "A". This puzzles me - why does the traffic shaping now make bufferbloat worse on my fibre connection, when it used to make things better on the XDSL connection?

My traffic shaping settings are:
Pipes
Down
  Bandwidth 430 Mbps
  CoDel not enabled
  (FQ-)CoDel ECN enabled
  FQ-CoDel quantum 1290
Up
  Bandwidth 290 Mbps
  CoDel not enabled
  (FQ-)CoDel ECN enabled

Queues
Down
  Weight 100
  mask destination
  (FQ-)CoDel ECN enabled
Up
  Weight 100
  mask source
  (FQ-)CoDel ECN enabled
Rules
Down
  Interface WAN
  Protocol ip
  Source any
  Src-port any
  Destination 192.168.2.0/24 (the OPNSense is 192.168.2.0 and hands out DHCP addresses 192.168.2.1-255)
  Direction both
  Target download queue
Up
  Interface WAN
  Protocol ip
  Source 192.168.2.0/24
  Src-port any
  Destination any
  Direction both
  Target upload queue

I could just leave the traffic shaping turned off and forget about it, but I would like to understand what is going on in case I need to turn it on in the future.

[jrichey98]

I have ATT 500mbps symmetric fiber.

Pipe: ATT-IN
Bandwidth: 540
Scheduler Type: fq_codel
Target: 2ms
Interval: 4ms
Quantum: 1514 (Packet Size, see https://man.freebsd.org/cgi/man.cgi?ipfw)
Limit: 1200 (about perfect for 500mbps)

Pipe: ATT-OUT
Bandwidth: 540
Scheduler Type: fq_codel
Target: 2ms
Interval: 4ms
Quantum: 1514 (Packet Size, see https://man.freebsd.org/cgi/man.cgi?ipfw)
Limit: 1200 (about perfect for 500mbps)
ECN: Enabled

---
Queues:
- HIGH-IN/OUT
- LOW-IN/OUT
Rules:
-IN & OUT rules for pushing ACK packets under 140 bytes to HIGH.
-IN & OUT rules for pushing DNS to High (by IP) to HIGH.
-IN & OUT rules for pushing regular traffic to LOW.

Notes:
-For rules I use the direction parameter to specify in/out instead of source/dest.
-I only do ECN for outgoing packets, because windows and linux now support it, but not every router/system between you and the internet host probably will.
-Quantum should be large enough to fit a packet, limit should be scaled with bandwidth.
-Don't be afraid to play with super low target/intervals with fiber if appropriate. It could be pretty low latency to your first hop. Do a ping to the first gateway and set your target/interval appropriately.

Results: https://www.waveform.com/tools/bufferbloat?test-id=1ece2bf6-b7a2-4a40-9817-c90db7eafcd1
Best: 13ms+0+0,
Any time every day: 14ms+1+1 / 15ms+0+0