Traffic Shaper weighted queues, should I use "mask"?

[rdr]

Hi,

I am new to OPNSense. I am trying to shape our internet connection so traffic is more evenly processed and I can prioritize important traffic. We have 500/500 mbps symmetric fiber connection, which we see saturated download at work hours.

I read documentation and I have some question. I do not want to set hard limits on any traffic so any one can use bandwidth if no body use it. I also want to set higher priority for some traffic and I also want to share available traffic inside queues.

1. Documentation has examples, it says I need to use "mask" on the queues to evenly share bandwidth inside queue. But the next example about "weighted queues" says to use "empty" mask. So What should I do? I need to have 3-4 queues that points to the same download pipe ( witch I set %10 lower than actual line speed, 450 mbps).

Queue A = Weighted 1 for catch all download traffic
Queue B = Weighted 100 for important traffic (DNS queries, NTP traffic etc etc)
Queue C = Weighted 80 for prioritize traffic for some other traffic (Some government websites)

So Should I use "mask" for this queues? If I use "mask", it creates seperate flows for all clients downloading. Will these clients share parent queues "weight"? If there are 1000 flows in Queue A, will each flow has weight 0.001 or they will have weight 1 each? If they will have separate weight then they will overrun the weight 100 queue as they will have very few flows?

If I don't use "mask" queues will have single flow but then is it mean bandwidth will not shared inside queue evenly?

2. Should I enable Codel? If yes, should I enable it on the pipe itself or on the queues or on both?

3. For our situation, should I increase slots (defult 50) or buckets(default 256)? With using mask, when we have +1000 flows inside Queue A, I see 4-5 client connection share same bucket. Is this a problem? Should I set bucket size more that expected flows?

I will be very glad if someone can help me on this.

Regards,

Rahman

[gpb]

Good questions...I have no idea what the answers are.  Apparently along with the incredible flexibility of the shaper makes it equally complex.  If you haven't already, you might check this link for some additional information.

https://www.freebsd.org/cgi/man.cgi?query=ipfw&sektion=&n=1

Scroll down to the section "PIPE, QUEUE AND SCHEDULER CONFIGURATION", about half-way...just do a page search.  At least you'll have some definitions, not sure if it will ultimately be helpful or not.

Hopefully someone that has experience provides some insight into your questions.  Cheers.

[mimugmail]

Dont use mask, just use Fq_codel as scheduler and use queues where important gets 10, not so important 5, rest 1. If there is only traffic for 1 it still gets full BW

[rdr]

Dont use mask, just use Fq_codel as scheduler and use queues where important gets 10, not so important 5, rest 1. If there is only traffic for 1 it still gets full BW

I think I read somewhere that FQ_codel scheduler has no weight meachanism, only wfq+ respects queue weighs? So what do you suggest, use FQ_codel scheduler instead of wfq+ on the pipe and continue to use seperate weighted queues?

[mimugmail]

It could be me that stated this, but then I was wrong since its called FairQueue_CoDel

[gpb]

Sorry to interject, but I think @rdr did read it correctly.  FQ = FlowQueue according to the link above.  Looks like QFQ is also an option for weighted queues...not sure the pros/cons.

Code Select Expand


     fifo    is just a FIFO scheduler (which means that all packets
     are stored in the same queue as they arrive to the sched-
     uler).  FIFO has O(1) per-packet time complexity, with
     very low constants (estimate 60-80ns on a 2GHz desktop
     machine) but gives no service guarantees.
     wf2q+   implements the WF2Q+ algorithm, which is a Weighted Fair
     Queueing algorithm which permits flows to share bandwidth
     according to their weights.  Note that weights are not
     priorities; even a flow with a minuscule weight will
     never starve.  WF2Q+ has O(log N) per-packet processing
     cost, where N is the number of flows, and is the default
     algorithm used by previous versions dummynet's queues.
     rr      implements the Deficit Round Robin algorithm, which has
     O(1) processing costs (roughly, 100-150ns per packet) and
     permits bandwidth allocation according to weights, but
     with poor service guarantees.
     qfq     implements the QFQ algorithm, which is a very fast vari-
     ant of WF2Q+, with similar service guarantees and O(1)
     processing costs (roughly, 200-250ns per packet).
     fq_codel
     implements the FQ-CoDel (FlowQueue-CoDel) scheduler/AQM
     algorithm, which uses a modified Deficit Round Robin
     scheduler to manage two lists of sub-queues (old sub-
     queues and new sub-queues) for providing brief periods of
     priority to lightweight or short burst flows.  By de-
     fault, the total number of sub-queues is 1024.  FQ-
     CoDel's internal, dynamically created sub-queues are con-
     trolled by separate instances of CoDel AQM.
     fq_pie  implements the FQ-PIE (FlowQueue-PIE) scheduler/AQM algo-
     rithm, which similar to fq_codel but uses per sub-queue
     PIE AQM instance to control the queue delay.

[rdr]

It could be me that stated this, but then I was wrong since its called FairQueue_CoDel

Yes it should be you stated that :) But read lots of dumynet/ipfw documentation and if I understand correctly, they say FQ_codel scheduler does not respect any ipfw/dumynet sub queues, it creates and uses its own queues. So it should not behave with weights of ipfw/dummynet queues?

[rdr]

Good questions...I have no idea what the answers are.  Apparently along with the incredible flexibility of the shaper makes it equally complex.  If you haven't already, you might check this link for some additional information.

https://www.freebsd.org/cgi/man.cgi?query=ipfw&sektion=&n=1

Scroll down to the section "PIPE, QUEUE AND SCHEDULER CONFIGURATION", about half-way...just do a page search.  At least you'll have some definitions, not sure if it will ultimately be helpful or not.

Hopefully someone that has experience provides some insight into your questions.  Cheers.

Thanks for the link, I read this and also read pfsense docs about dummynet. I could only get answer to my first questin :)

* Using mask causes dynamic copies of the parent queue with the "same" weight. So lets say I create a queue with ip mask and weight 1 and second queue with ip mask weight 100. Then first queue has 2000 flows and second queue has 10 flows. As I understand, then the PIPEs bandwidht will be shared as 2000x1=2000 for first queue and 10x100=1000 for second queue. As you see I want to to have 100x more priority for second queue but instead it will have half priority of the first queue in the end.

* Without using mask, all traffic from different IPs will end up in the same flow so they are not guarantined to get even bandwith. It is up to the scheduler.

* I could not learned about WFQ+ with codel enabled on the pipe or queue will respect to queue weights or not.

* as I do not use mask, bucket size is not important for me. But I could not learned about bucket size etiher.

[gpb]

I've been trying to tweak QoS just on the basis of buffer bloat testing.  That ultimately led me down the path of trying to understand how the shaping works.  Some/most changes don't seem to affect results, others clearly do.  Not using a mask on pipe doubles latency values, for example.  But I really don't know if I've configured things properly or not.  I'm pretty happy with the where I"m at, full bandwidth is used while not having too much overhead.

Results for approx 230/12 service (DOCSIS):
https://www.waveform.com/tools/bufferbloat?test-id=21b5db30-743d-4e6e-a567-a93a1091cf01

I'm sure you probably looked already, but there is a status option under shaping in the UI which can show some additional data to see if rules are working as you expect.  Good luck.

[klaberte]

I think my understanding mirrors your own.  I wish to make the following clarifications:

• Both pipes and queues can have masks
• If the pipe or queue has Mask=[None], it creates a single pipe or queue
• If the pipe or queue has any other Mask, it creates multiple pipes or queues, where each Mask value creates its own pipe or queue
• Generally a pipe is used to limit a specific resource, such as a maximum amount of bandwidth.
• For example, if you have a pipe limited to 1MB, and it has Mask=None, then only a single 1MB pipe is created by that definition.
• However, if you have a pipe limited to 1MB, and it has any other Mask value, multiple 1MB pipes, one for each Mask value, can be created.
• Generally, queues must be assigned to a specific pipe.
• Generally, if you wish to give a flow a specific weight, you use queues.  These weights are used to indicate how the pipe is to be shared across its queues

Good questions...I have no idea what the answers are.  Apparently along with the incredible flexibility of the shaper makes it equally complex.  If you haven't already, you might check this link for some additional information.

https://www.freebsd.org/cgi/man.cgi?query=ipfw&sektion=&n=1

Scroll down to the section "PIPE, QUEUE AND SCHEDULER CONFIGURATION", about half-way...just do a page search.  At least you'll have some definitions, not sure if it will ultimately be helpful or not.

Hopefully someone that has experience provides some insight into your questions.  Cheers.

Thanks for the link, I read this and also read pfsense docs about dummynet. I could only get answer to my first questin :)

* Using mask causes dynamic copies of the parent queue with the "same" weight. So lets say I create a queue with ip mask and weight 1 and second queue with ip mask weight 100. Then first queue has 2000 flows and second queue has 10 flows. As I understand, then the PIPEs bandwidht will be shared as 2000x1=2000 for first queue and 10x100=1000 for second queue. As you see I want to to have 100x more priority for second queue but instead it will have half priority of the first queue in the end.

* Without using mask, all traffic from different IPs will end up in the same flow so they are not guarantined to get even bandwith. It is up to the scheduler.

* I could not learned about WFQ+ with codel enabled on the pipe or queue will respect to queue weights or not.

* as I do not use mask, bucket size is not important for me. But I could not learned about bucket size etiher.

[muchacha_grande]

May be you already did it, but if you didn't, take a look at this page.

https://docs.ibracorp.io/opnsense/

I had shaper configured and working fine before doing this how to. But since I adjusted some params as this page points out the shaper is doing better.

One of the parameters that I touched is the mask on pipes and on queues. Before I was not using mask and now I am.