CoDel/Shaper Maxing Out at 750 Mbps?

[EFaden]

I have a 1000Mbps/20Mbps cable connection.  When I connect directly to the modem I am able to get the full speed, so I know the line works.  When I run the speed test with my current codel config I max out download at 750Mbps and I can't seem to figure out why.

I have a Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz w/ 16 GB RAM and the following setup....

Everything not marked below is either blank/unchecked/default etc

Pipes:
  1) Speed=1000, Units=Mbps, Scheduler=FlowQueue-CoDel "Downstream Pipe"
  2) Speed=20, Units=Mbps, Scheduler=FlowQueue-CoDel "Upstream Pipe"

Queues:
  1) Pipe=Downstream Pipe, Weight=100, "Downstream Queue - Regular"
  2) Pipe=Upstream Pipe, Weight=1, "Upstream Queue - Regular"
  3) Pipe=Upstream Pipe, Weight=10, "Upstream Queue - DNS and ACK"

Rules:
  1) Sequence=1, Interface=WAN, Proto=IP, Source=Any, Dest=<Private IPS>, Src/Dest Port=ANY, Direction=Both, Target=Downstream Regular
  2) Sequence=10, Interface=WAN, Proto=TCP ACK ONLY, Source=<Private IPs>, Dest=ANY, Src/Dest Port=ANY
 Direction=Both, Target=Upstream Queue - DNS and ACK
  3) Sequence=11, Interface=WAN, Proto=TCP, Source=<Private IPs>, Dest=ANY, Src Port=Any, Dest Port=53
 Direction=Both, Target=Upstream Queue - DNS and ACK
  1) Sequence=12, Interface=WAN, Proto=IP, Source=<Private IPS>, Dest=Any, Src/Dest Port=ANY, Direction=Both, Target=Upstream Regular

My goal is to have codel setup to stop buffer bloat issues, and also to prioritize ACK and DNS.  But I can't figure out why when these are enabled my connection maxes out at 750

[mimugmail]

What happens when you give Download Pipe 1500 Mbit?

[EFaden]

No real difference.  With everything disabled I get 900 to 960/20 which is basically what I get with my laptop directly connected.  With enabled I get 750/20.  I changed download pipe to 1500 and still get about 750/20.

[EFaden]

I will say this is somewhat interesting.  I played around with disabling and enabling different rules and I got a result that is confusing.  With the download rule enabled but the upload disabled my download speedtest goes to 860-900/20.....  But with download queue disabled and upload enabled I get 750/20.

I'm not really sure I can explain that.  So it seems like the upload rules are causing a problem for the download test?

I simplified the rules to just

  1) Sequence=1, Interface=WAN, Proto=IP, Source=Any, Dest=<Private IPS>, Src/Dest Port=ANY, Direction=Both, Target=Downstream Regular
  2) Sequence=12, Interface=WAN, Proto=IP, Source=<Private IPS>, Dest=Any, Src/Dest Port=ANY, Direction=Both, Target=Upstream Regular

And it limits.  If I disable the rule 2 (e.g. upload) I get the normal speed.  If I change my upload pipe to 50 Mbps it seems to resolve.  But I don't really understand why.  And changing my upload pipe to above my true upload speed seems to defeat the purpose. I'm honestly very confused.

[mimugmail]

I remember an issue in GitHub that matching ACK packets needs also some size tuning in rule. Maybe your acks doesnt match and your download wont speed up due to late acks?

[EFaden]

Interesting idea. 

I found https://github.com/opnsense/core/issues/4132

[mimugmail]

Exactly, give it a spin please. Maybe docs need also an update then

[EFaden]

Tried messing around with the len.  Didn't seem to change anything.