[Tutorial] - How to configure fq_codel for comcast to help bufferbloat / QoS

[meyergru]

The ECN standard is pretty new in networking periods of time, so not everybody (i.e. ISP) supports it properly.

Using it can have adverse effects because some router equipment may abort the connection if it sees the previously reserved bits that are now being used for ECN. That is why I disabled it after seeing problems with traffic shaping with ECN enabled, alas to no avail.

So, there is no real suggestion. Theoretically, ECN would allow to signal that a congestion is imminent, so that is a plus. But if it is implemented poorly or not at all on your ISP's side, it is counter-productive.

[Seimus]

Basically I agree what @meyergru said.

There are no clear instructions on this, you can enable it and if you don't see any problems on the flows (sessions/connections) keep it enabled.

What does slow start refer to exactly and is there an easy way to test it?

Slow start is exactly what is says "slow start of the flow" "TCP slow start", meaning you download something but it takes a while to reach the throughput it should.

In regards of tests, there are several tools than can be used to test it scientifically. For example crusader. But as well speed test can show it, those one mentioned in the Docs guide, or Iperf3. Or you can see it on real live flows if you see the throughput values.

Regards,
S.

[neo243]

@theogravity Thanks for creating this works nicely!

[MagikMark]

I was wondering if we could prioritize acks inside the pipe to process this first as this may help with the latency in local network during heavy load

[Seimus]

I was wondering if we could prioritize acks inside the pipe to process this first as this may help with the latency in local network during heavy load

You can definitely create a separate Queue for ACK before it hits the Scheduler which creates internal queues per FLOW. This approach is good because when you have a single created Queue it can only fit 50 packets and than it TAIL drops.

However, you can not prioritize this Queue from the others. The weights in Queues when using FQ_C is not used, and FQ_C internal per flow Queues dont offer any prioritization either.

Regards,
S.

[MagikMark]

@Seimus

Thanks for the reply can you help with a guide on how I should proceed.  So far I experimented with the ff:

Pipes:

1.  Queue set to 2.  Found out this works the best specially when using VPN from the client side.  If I set this to default sometimes speed under VPN from client side do not burst.  It's stuck to 10 or 100Mbps.  My line is 600Mbps

Queue:

1.  Created a priority upload queue with weight of 10 and the rest is set to 1.

Rules:

1. Sequence 1 is for download.  Sequence 2 is for upload. Sequence 3 is priority upload.

Result.

Priority Upload is not getting any hit.  Changed the rules sequence by putting priority upload to first.  I'm now getting hits.  However, VPN speeds originating from clients sometimes do not burst.



*VPN is set using a TorGuard app in the client's machine.  It's NOT set in OPNsense

[Seimus]

You can follow the official docs and create more Queues + attach specific rules to them. Order of rules matters cause its going from top to bottom.
https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html

In regards

Pipes:
1.  Queue set to 2.

The Queue setting in Pipes is used for dynamical created Queues. This is used when you don't create manually the Queues under the Queue tab. FreeBSD & OPNsense give you the possibility to create Pipe + Rule only, however there always needs to be a Queue as Queue is an adapter. When you create the Queues manually, this setting still creates the dynamic Queues but they are not used; because you attach Rules to the specific Queues you manually created.

Queue:
1.  Created a priority upload queue with weight of 10 and the rest is set to 1.

As mentioned the Weight does nothing if FQ_C is used, Weighs are used by WFQ for example but not by FQ_C

Rules:
1. Sequence 1 is for download.  Sequence 2 is for upload. Sequence 3 is priority upload.

FQ_C doesn't provide possibility for prioritization as mentioned and why you can read more in here https://forum.opnsense.org/index.php?topic=43856.0

Another thing, I am not sure what do you mean by bursts, burst in networking e.g QoS is a time window that allows exceeding of the configured rate, this however is done by policers, shapers can not do that.

Regards,
S.

[MagikMark]

@Seimus

Thank you very much to your insights.  I really appreciate it.

My new experimental setting:

Pipe:

1.  Queue 3

Rules:

1.  added tcp ack rule on top of the other rules

Results

It looks like latency went down by 10ms.  Will be running this for a couple of days


* The bursting I meant was the speed we are getting from the ISP.  I'm on a 600Mbps plan and the speed burst between 200 to 600Mbps

[Seimus]

Its better to create manually the Queues attach them to the Pipes, and attach Rules to the manually created Queues. It will give you better control.

Order of configuration
Pipe(Scheduler) > Queue > Rule

Order of packet flow
Packet match > Rule > Queues > (Scheduler)Pipe

Order of packet flow in case of FQ_C
Packet match > Rule > Queues > Scheduler
                                                                 \Flows (per 5-Tuple) > Pipe


Also

* The bursting I meant was the speed we are getting from the ISP.  I'm on a 600Mbps plan and the speed burst between 200 to 600Mbps

This is called throughput not bursting.

Regards,
S.

[MagikMark]

@Seimus

There's a new Traffic Shaping feature in the Firewall Rule (experimental).  What would be a good practice for this?

[Seimus]

Currently that feature is experimental so take it with a grain of salt.

Basically what that feature does is to give you a possibility to set a Queue or Pipe directly in the rules of pf (Firewall > Rules) instead of ipfw (Sharper > rules). Which means you can use as well aliases for example and all the goodies that you can normally do in Firewall rules but not in Shaper rules. Or you can do application specific Shaping like for example you create a rule for HTTP/s and use the same rule in the Queue + Pipe. Thus you dont need to create a separate Rules in the Shaper section.

Personally I didn't yet try it out as I didn't had any need for it. But few use cases come to my mind like the one above.


Regards,
S.