FQ-Codel for bufferbloat reduces upload speed by 4

[marc.turcotte.qc]

Hello all,

I recently switched from a 60/10 Mbps cable connection to a 1/1 Gbps fiber connection and I need your help to figure out why my upload speed gets reduced by a factor of 4 when I configure a FQ-codel shaper.

My router has four 2.5 GbE ports (WAN, LAN, WIFI, VOICE) where each port is dedicated to a network with devices having the same role. The WAN port been obviously connected to the ISP modem (with PPPoE).

Multiple speed tests with the Ookla Windows app and a server as close (network-wise) as possible showed me a download speed of 946 Mpbs and an upload speed of 942 Mbps.

Multiple buffer bloat tests with waveform and a PC connected with a 1 GbE port on the LAN network showed me that I had a buffer bloat grade of “C” with an averaged download speed of 868 Mbps and an upload speed of 950 Mbps.

I wanted to bring the buffer bloat grade to an acceptable level. So I followed the Fighting Bufferbloat with FQ_CoDel — OPNsense documentation and Improve your BufferBloat with Traffic Shaping in Opnsense with IPv6 - Maltechx.de and managed to get an A+ score with a “shaper” configured with:
•   2 pipes with ECN (at 85% of the bandwidth measured with Ookla)
•   2 queues with ECN
•   2 rules

This is where it is getting weird…

A got A+ score with an averaged download speed of 735 Mbps but my upload speed is now limited to 199 Mbps.

A test with the Ookla windows app showed the same results.

Disabling and enabling only the upload shaper rule or both rules give me consistent results:
•   Disabled: Download 868 Mbps, Upload 950 Mbps
•   Enabled: Download 735 Mbps, Upload 199 Mbps

Reducing the Upload Pipe’s bandwidth to 600 results in an Upload speed of ~150. Reducing it further has the same effect.

I went to the Shaper’s Status page and enabled the “Show Rules” and “Show active flows” and noticed that no flows are shown when refreshing the page while running the Wafeform or the Ookla test.

That made me think that I had a configuration error, but I checked the “shaper’s” configuration several times and I just can’t find any error.

Do you see what I'm doing wrong?

[marc.turcotte.qc]

Download config

[marc.turcotte.qc]

Upload config

[marc.turcotte.qc]

I forgot to mention that I use OpenSense 24.7.1-amd on an Intel j4125 based router.

[Koldnitz]

I think you are doing it wrong.

Some links:

https://docs.ibracorp.io/opnsense

Seimus speaks sense:

https://forum.opnsense.org/index.php?topic=39046.0

As does Fenuxx (I'm an amateur):

https://forum.opnsense.org/index.php?topic=39046.15

White paper I bookmarked but don't totally understand:

https://datatracker.ietf.org/doc/html/rfc8290

I hope this helps.

Cheers,

[AhnHEL]

I have the same config as you with some minor differences in the bandwidth speeds and I don't use ECN within the Queues.  ECN is only enabled on the Pipes.  I'm not seeing the symptoms you're seeing though.  How's your CPU usage when running these speedtests?

[Seimus]

The docs are the correct way for how to configure FQ_C on OPNsense. This was created in collaboration with the Bufferbloat community. I tested this guide with 2 other people on 2 different devices and providers using OPNsense before it was committed to the Docs. None of those "testers" reported issues like you have.

Whole reason for the docs, were to have one proper source of truth as a lot of misinterpretation/missconfiguration of the FQ_C were going on the forum and Internet. And as well to create as possible simplest configuration for the general user.

The ECN doesn't need be used on the Queue, as mentioned in the docs, these parameters are specifically only for CoDel and not FQ_C, as FQ_C is set in the scheduler where CoDel is set in the Queue.

(Check the NOTE)
https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html#create-queue-for-download

You will not see any flows in the FW > Shaper > Status, this is because the flows created per 5-tuple are visible only for the FQ_C algorithm, where it puts into specific dynamic queues. So this is normal.


As advised by @AhnHEL, check your CPU and load utilization during running speedtest. Shaper can be CPU taxing. Also try to disable ECN completely off (in scheduler > Pipe). It can in certain situation cause problems as mentioned in the docs.

https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html#ecn

P.S. The rules in the docs should cover IPv4 as well IPv6, as its set to IP any/any

Regards,
S.

[doktornotor]

FWIW, with exact the setup sans ECN on pipes, I can get the full upload speed on a ~250/50 VDSL line. HW not capable is my guess, as others suggested.