Bufferbloat tests lie to me.

[Siarap]

Ive tested bufferbloat here: https://www.waveform.com/tools/bufferbloat with A+ result and here https://speed.cloudflare.com/ with result: great (top result) . But when im downloading with full speed and pinging some domains in my country i get ping over 750+ ms (its variable up to this value). I dont understand why this is happening. Im currently using fq codel setup from opnsense docs: https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html

Why this tests lie?

[sja1440]

There are several aspects to this. here are a few that come to mind:

(1) The two test sites that you mention (waveform and cloudflare) do not use icmp/icmpv6 "echo requests" (i.e. classic ping) but measure the time difference between a request for data and the receipt of the data using tcp/udp. Cloudflare explicitely says so. To convince yourself, you could also do a wireshark capture - there will be no icmp/icmpv6 echo requests. I believe that it is also true that internet routers in general treat icmp/icmpv6 echo requests differently (in terms of priority and treatment) to other protocols.  So the high "ping" latency you see is not really relevant.

(2) Did you tune the pipe bandwidths of your configuration? At somepoint in the ISP infrastructure you will be sharing bandwidth with other users so it is important to test at different times of day. For example, I have a 2.5Gbps/1Gbps fibre connection but in order to get A+ at all times of day, I need to set my bandwidths to 1.5Gbps/600Mbps - at 60% this is way below the article's suggested 85% starting point. If I try to increase the bandwidths, my latency invariably rises quickly in the evening.

[Siarap]

Thanks for explanation. Cutted my bandtwith by about 20% currentlu it works best for me. Even when i cut more or less it has worse results on my docsis connection . Really weird situation.

[Seimus]

The 85% start point in the docs, is basically an average expected value. This doesn't mean it applies to every use-case or circuit. Thats why tuning the BW in Pipe is important and its pointed out to be one of the most important parameters to tune UP or DOWN.

You maybe are wondering when you own a high speed connection that docs say 85% to start with, and end up less with 85%. This is sadly to be expected, the reason behind this is your provider. Basically your ISP over-provisions your connection, they are maybe able to deliver the advertised speed on the last mile e.g. from their GW directly to your device or their concentrator, but, somewhere on the PATH they have a bottleneck be it a Capacity bottleneck or a device that can not handle the amount of packets aggregated by all customers.


Now in regards of testing sites for buffer bloat. They do not show the whole picture, the tests are static and can not cover all data-traffic patterns a user can generate. Its a good start point and usually should cover you good. But for example when I was tuning Shaper for RTP such as IPTV I had a stream opened and run speedtest while tuned the parameters in Shaper till I didn't get it to a point where the stream stopped to stutter and lag.

The outcome is, while my Internet connection can be saturated or something in the ISP path may be saturated and ping can show higher latency, the stream is stable.

Regards,
S.

[jackjohnson]

Ive tested bufferbloat here: https://www.waveform.com/tools/bufferbloat iq test free with A+ result and here https://speed.cloudflare.com/ with result: great (top result) . But when im downloading with full speed and pinging some domains in my country i get ping over 750+ ms (its variable up to this value). I dont understand why this is happening. Im currently using fq codel setup from opnsense docs: https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html

Why this tests lie?

Great job on the A+ bufferbloat test! However, high ping (750+ ms) while downloading could be due to network congestion or QoS settings. Even with fq_codel, full-speed downloads can spike latency. Try adjusting your download speed to see if it helps stabilize your ping.

[meyergru]

Sorry, but you are a little late to the party.

Shortly after the initial post in this thread, some fixes to the traffic shaper have been implemented and we found that control plane shaping does the trick, which is now even in the docs: https://docs.opnsense.org/manual/how-tos/shaper_control_plane.html

So, there are basically two answers to the original question:

1. Do not use up all the available bandwidth as the limit, since there will be no more headroom left for shaping.
2. Implement control plane shaping that gives a higher priority to traffic that needs it. In this case, you would need to implement separate queues for ICMPv4 and ICMPv6.

[Seimus]

I would like to just point here,

Control plane QoS/Shaping, is here to guarantee stability of the network during congested state, not to improve for example ping latency.

When control plane QoS/Shaping is configured, it should be always done for the network devices that are responsible for maintaining the stability of the network. Thus with precise specified sIP dIP +/or protocol specific. But due to the fact the documentation needs to account for a larger audience, it was written in more generic way, so even a casual user can understand and implement it.

Regards,
S.