Feature Request: Traffic shaping with PRIQ

[Mr.Goodcat]

As far as I can see this feature is currently only available on PFSense, but having it available on OPNSense as well would of course be prefered 

Many internet connections do not provide stable data rates (DOCSIS I'm looking at you). Hence, it would be preferable if one could clasiffy traffic into different queues and have them served depending on their priority. For example:
Q0: SSH
Q1: VOIP
Q2: Gaming
Q3: Video Streaming
Q4: Anything else

The scheduler should then always give whatever bandwidth is required to Q0 if there is traffic. Any remaining data rate can be used by Q1. Whatever is still unused would be available to Q2 and so on. Thereby, the lowest priority services might be cut off entirely if there is just enough capacity for higher priority services.

https://de.slideshare.net/NetgateUSA/traffic-shaping-basics-with-priq-pfsense-hangout-february-2016

Keep up the great work and merry Christmas! 

[mimugmail]

PRIQ depends on ALTQ which is not supported in OPNsense,  sorry

[Mr.Goodcat]

What does OPNSense use? To my knowledge ALTQ is BSDs default scheduler, and OPNSense is based on FreeBSD. So it should be possible to port this?

EDIT: Found something:

The framework behind the "limiter" tab in pfSense is essentially what we have picked as our shaper technology. All other parts based on ALTQ were removed, mostly because ALTQ is disabled in FreeBSD GENERIC builds and also because OpenBSD removed ALTQ in favour of a directly plugged HFSC shaping algorithm. ALTQ was thought of as a way to deliver many shaping technologies, but over the years (at least for OpenBSD) only HFSC came to matter.

ALTQ is directly plugged into pf(4), while the limiter technology based on ipfw(4) and dummynet(4) runs as a second completely detached packet filter. This brings a few limitations: you cannot use pf(4) rules to shape traffic anymore, this is an important detail, because the filtering in ipfw(4) is not as advanced (it has the day to day basics but not such things as e.g. OS detection).

This also brings an advantage: when disabling the firewall, you can still shape the traffic for routing...

dummynet(4) used to misses the CoDel algorithm which pfSense ALTQ has, but it recently became available in a first version, which we have picked up already and will provide GUI support for in a couple of weeks[1].

[1] https://lists.freebsd.org/pipermail/freebsd-ipfw/2016-February/006026.html

The main difference is in configuration approach (queues, pipes, rules) and in the rules complexity itself. In the majority of use cases, the missing rules flexibility does not matter.

This is just a technical overview. Others can tell more about the shaper differences from an actual user perspective or how they are using the shaper.

Now the question is: how could a simple prioritization be implemented with this? Becuase the fact, that many internet connections just don't provide stable datarate, remains.

[mimugmail]

There's atm no way to achieve this in a stable manner.

[guest15389]

I just use a FQ-Codel Pipe with 3 queues with Weights of 100/25/1.

I drop all my high priority traffic in the 100 queue (VOIP/Gaming) and my torrenting in the 1 Queue. Defaults to everything else in the 25 queue.

Works like a charm as my 1 weight uses up everything it can and if something more important comes along, it gets plenty of priority.

[mimugmail]

Good idea! QFQ scheduler is also nearly perfect for this .. but both is not REAL priority queueing.
But no user should feel the difference as it is very well implementated

[abraxxa]

@Animosity022: can you please document your setup in more detail as I'm trying to replicate the same thing but failed so far, thanks!

[mimugmail]

@Animosity022: can you please document your setup in more detail as I'm trying to replicate the same thing but failed so far, thanks!

1. Create two pipes, with traffic definition for inbound and outbound, use scheduler fq_codel
2. Create six queues, three for in and out. Each of three gets weighting of 100, 25 and 1.
3. Create six rules, three with direction IN on WAN and three with OUT on WAN. Then match via src/dst and link it to the specific queue.

[abraxxa]

Thanks!
That‘s almost exactly what I‘ve configured.
I don‘t have checked the CoDel or PIE checkboxes in either the queues nor the pipes, still the status output shows fq_codel. Do I have to enable one of those in one of the teo locations?
How should the status output look like?

[mimugmail]

If you use FQ_CoDel, never tick CoDel or PIE anywhere .. how do you check that it's not working?

[abraxxa]

Ok.
I checked by having two downloads, one per queue and checking their speeds.
Status only shows a single line in the last section. I can‘t access the firewall from here, will post the output when at home.

[abraxxa]

This is the status output:

Code: [Select]


Limiters:
10000:  10.000 Mbit/s    0 ms burst 0
q75536  50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail
 sched 75536 type FIFO flags 0x0 0 buckets 0 active
10001:  40.000 Mbit/s    0 ms burst 0
q75537  50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail
 sched 75537 type FIFO flags 0x0 0 buckets 0 active


Queues:
q10004  50 sl. 0 flows (1 buckets) sched 10001 weight 10 lmax 0 pri 0 droptail
q10002  50 sl. 0 flows (1 buckets) sched 10001 weight 100 lmax 0 pri 0 droptail
q10003  50 sl. 0 flows (1 buckets) sched 10000 weight 10 lmax 0 pri 0 droptail
q10001  50 sl. 0 flows (1 buckets) sched 10000 weight 100 lmax 0 pri 0 droptail


Schedulers:
10000:  10.000 Mbit/s    0 ms burst 0
q75536  50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail
 sched 10000 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 10003 10001
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0        4     4696  0    0   0
10001:  40.000 Mbit/s    0 ms burst 0
q75537  50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail
 sched 10001 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 120 limit 120 flows 1024 ECN
   Children flowsets: 10004 10002
  0 ip           0.0.0.0/0             0.0.0.0/0        3      120  0    0   0


I was expecting the four queues to show flow counters not equal zero or even the individual flows.

My configuration consists of two pipes (40MBit down, 10MBit up), four queues (two down, two up, each a pair of normal and low prio traffic with a weigth of 10 (low prio) and 100 (normal prio)), and four rules matching the traffic. All are assigned to the WAN interface, the low prio (6 and 7) are before the normal prio rules (11 and 21) and match on the source and destination port only.

[mimugmail]

How do the rules look like?

[abraxxa]

For example the low prio inbound rule:

Code: [Select]

sequence: 6
interface: WAN
proto: IP (i'm dual stacked)
source: any
invert source: unchecked
src-port: any
destination: any
invert destination: unchecked
dst-port: 12345
target: low-prio download queue

[mimugmail]

Ok, and the weighting goes wrong or whats the problem with it?