Traffic shaping rules not being applied (23.7.9)

[MeltdownSpectre]

Hi,

I recently upgraded my ISP plan from 100/100 to 150/150.

I have traffic shaping with FQ-CoDel enabled on my main interface called 'Home', based on this tutorial.

https://forum.opnsense.org/index.php?topic=7423.0

Works like a charm, I get A+ bufferbloat and lag free online gaming.

I also had bandwidth limits of 20 Mbit/s down and 10 Mbit/s up on my Guest interface.

Instead of editing the existing rules, I deleted the old ones and added new bandwidth limits for the guest interface with 30 Mbit/s down and 15 Mbit/s up. However, no matter what I try, the rules aren't getting applied and the 'Status' tab under Firewall Shaper doesn't show the rules for the guest interface.

I followed this (previously) for setting bandwidth limits on the guest interface, and it worked fine.

https://docs.opnsense.org/manual/how-tos/guestnet.html

If I run a speedtest while connected to my Guest network, I get the full 150 Mbit/s up and down, so the rules aren't being applied.

Screenshots attached.

Currently running 23.7.9, and have tried rebooting multiple times after applying the shaper rules but it makes no difference.

[iMx]

Would really need to see the Pipe, Queues and Rules - with Advanced Mode toggled - themselves.  Something is certainly not matching on the upload rules, possibly Queue mask is wrong/not set, or Rule direction wrong or not set.

However, FQ - Fair Queue, or Flow Queue - is not really meant for doing 'multitier' on 1 connection - it's for 'fair' sharing, all traffic, amongst all flows, on 1 pipe.

With 2 pipes, 150/150 and 30/15, the firewall expects the total bandwidth available to be 180/165 - i.e there is no overlap and/or subtracting the 30 guest from the 150 total limit.

Whilst it's true that it is possible to restrict the Guest to 30/15, if you're running your main pipe at capacity (150 download, 30 on the guest, simultaneously) it's going to be over saturated.

P.S. There is no source/destination subnet set on the guest rules, also cannot see 'direction' there as well, nor mask on Queue.

[MeltdownSpectre]

Would really need to see the Pipe, Queues and Rules - with Advanced Mode toggled - themselves.  Something is certainly not matching on the upload rules, possibly Queue mask is wrong/not set, or Rule direction wrong or not set.

However, FQ - Fair Queue, or Flow Queue - is not really meant for doing 'multitier' on 1 connection - it's for 'fair' sharing, all traffic, amongst all flows, on 1 pipe.

With 2 pipes, 150/150 and 30/15, the firewall expects the total bandwidth available to be 180/165 - i.e there is no overlap and/or subtracting the 30 guest from the 150 total limit.

Whilst it's true that it is possible to restrict the Guest to 30/15, if you're running your main pipe at capacity (150 download, 30 on the guest, simultaneously) it's going to be over saturated.

Thanks for the explanation.

It's weird though, because I had 2 pipes of 100/100 before and 20/10 and it worked just fine. Guests were limited to 20 down and 10 up, and I'd still get 100/100 on my 'home' VLAN.

Here's an imgur link since the forum only allows 4 attachments per post.

https://imgur.com/a/uHw0ENe

[MeltdownSpectre]

P.S. There is no source/destination subnet set on the guest rules, also cannot see 'direction' there as well, nor mask on Queue.

I found conflicting information all over the internet and OPNsense documentation.

Some articles tell you to specify mask in traffic shaper rules and some say to leave it blank. The article linked in my main post says to set mask as 'Destination' for both download and upload pipes, whereas this article says to leave mask empty.

https://docs.opnsense.org/manual/how-tos/shaper_guestnet.html

[iMx]

For completeness, here are mine.

I disagree with a lot of the 'how to' out there, in the following:

- Firstly, I recommend reading the RFC.  It's nicely written and fairly easy to understand, it gives a better understanding of how things can/should be tweaked, than any How To:

https://datatracker.ietf.org/doc/html/rfc8290

- Quantum should not be adjusted, for high bandwidth general use. The default MTU 1500 +14 is 'good'.  The whole '300 per 100Mbits' has been completely misunderstood and is just plain wrong in my view.  Read the RFC. Tweak it below 100Mbits, above don't bother. If you have lots of small packets, maybe, otherwise don't bother.

- Don't change the queues on the pipes, leave them dynamically assigned.
- On the pipe FQ Codel Limit needs to be the maximum simultaneous packets, both directions, managed by the shaper instance.  The value in my screenshots, is the maximum allowed.
- FQ Codel flows, around the maximum states/sessions the firewall is expected to handle, i.e flows
- Delay of 1ms on the pipe, to stop out of order packets at saturation due to broken (?) fastio:

https://redmine.pfsense.org/issues/11192

- I use FQ Codel ECN, on the Pipe and Queue

Attachments to follow.

[iMx]

Up Pipe

[iMx]

Down Queue

[iMx]

Up Queue

[iMx]

Down Rule

[iMx]

Up Rule

[MeltdownSpectre]

I made some changes to my 150/150 rules, and bufferbloat is still at A+ thankfully.

But based on your post about the firewall expecting  me to have 180 Mbit/s total available bandwidth (150 for main and 30 for guest) I still don't understand how my guest bandwidth limits were working before....

[iMx]

I think the first question(s) that should be answered:

Do you really need to limit the guest network? Do you have guests that often, that heavily use your bandwidth? Do you have 'things' in your Home network that would really be that impacted by a guest downloading something - even if total bandwidth was 50/50 shared?

For me, the answer is a resounding 'no' - and simplicity is the best approach - I just want to make sure everything has a fair shot at the available bandwidth, without 1 flow taking all available bandwidth, to stop pipe saturation and hopefully bloating.

Your earlier screenshots are confusing FQ Codel and Weighted Fair Queue - FQ Codel is not and cannot be weighted and if you've only got 1 set of traffic weighted, then it's not weighted against anything else anyway.

I would suggest the following approach:

- Get everything working, i.e shaped, with 2 pipes, 2 queues, 2 rules, impacting everything
- Rules applying to just 1 interface, i.e WAN.
- Then, if you really want to, duplicate the above and modify the rules so that the source/destinations match for the home/guest subnets using FQ Codel (not weighted). Again, just on WAN interface.
- Whilst there is nothing stopping you from having 150/150 and 30/15 pipe, at the point of the link/connection being saturated the firewall thinks you have more bandwidth than you do unless you:

Main pipes 120/135
Guest pipes 30/15

.. even then, usually, you set your pipes to 90-95% of total available bandwidth.

Or.... ditch FQ Codel and do everything as weighted fair queue, then you can weight traffic, queues, etc.

[MeltdownSpectre]

I think the first question(s) that should be answered:

Do you really need to limit the guest network? Do you have guests that often, that heavily use your bandwidth? Do you have 'things' in your Home network that would really be that impacted by a guest downloading something - even if total bandwidth was 50/50 shared?

Spoiled brat kids who come over with one iPad each tend to sit and update every app in existence, all simultaneously so I'd rather they didn't have access to the full 150.

I would suggest the following approach:

- Get everything working, i.e shaped, with 2 pipes, 2 queues, 2 rules, impacting everything
- Rules applying to just 1 interface, i.e WAN.
- Then, if you really want to, duplicate the above and modify the rules so that the source/destinations match for the home/guest subnets using FQ Codel (not weighted). Again, just on WAN interface.
- Whilst there is nothing stopping you from having 150/150 and 30/15 pipe, at the point of the link/connection being saturated the firewall thinks you have more bandwidth than you do unless you:

I guess me foolishly trying to combine FQ Codel and WFQ earlier (without fully understanding them both) was causing some weird behaviour.

Tried your approach and all is well now. Got a great bufferbloat result as well:

https://www.waveform.com/tools/bufferbloat?test-id=1730cbc7-8534-4d22-9816-d5194bec2116

Thank you very much for your help.

[iMx]

Spoiled brat kids who come over with one iPad each tend to sit and update every app in existence, all simultaneously so I'd rather they didn't have access to the full 150.

Ah - I don't have that problem with Dogs ;)

No problem, pleased you got something that works.