Messages

1

General Discussion / Re: fq_codel latency spikes

« on: May 22, 2019, 10:31:34 pm »

The ethernet link to my Metro-E switch port was set to auto-negotiate on both sides and had negotiated half-duplex 100 Mbps. Switched to 100 Mbps full-duplex. Frontier engineer also removed shaping on my link. Problems seem to have gone away. Wish I knew which change made the difference, but I am extremely happy to have a resolution. I now get the best results with  the recommended settings for a 10 megabit link, and an 8.5 megabit pipe limit:

Code: [Select]

10000:   8.500 Mbit/s    0 ms burst 0
q75536  50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail
 sched 10000 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 1ms interval 3ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 10002 10000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0        1     1500  0    0   0
10001:   8.500 Mbit/s    0 ms burst 0
q75537  50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail
 sched 10001 type FQ_CODEL flags 0x0 0 buckets 0 active
 FQ_CODEL target 100ms interval 5ms quantum 300 limit 600 flows 1024 NoECN
   Children flowsets: 10001 10003

Additional practical testing will be required to verify that it is indeed fixed, but I am optimistic!

muchacha_grande, thank you so much for taking the time to help me. Having a second set of eyes look my rules over made me a lot more confident about calling Frontier for help.

2

General Discussion / Re: fq_codel latency spikes

« on: May 22, 2019, 08:07:53 pm »

I'm seeing *some* traffic in the scheduler:

Code: [Select]

# ipfw sched show
10000:   8.500 Mbit/s    0 ms burst 0
q75536  50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail
 sched 10000 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 1ms interval 3ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 10002 10000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0       58    85552  2 3000   1
10001:   9.000 Mbit/s    0 ms burst 0
q75537  50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail
 sched 10001 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 75ms quantum 1514 limit 10240 flows 1024 NoECN
   Children flowsets: 10001 10003
  0 ip           0.0.0.0/0             0.0.0.0/0        8      320  0    0   0


3

General Discussion / Re: fq_codel latency spikes

« on: May 22, 2019, 08:04:02 pm »

Code: [Select]

# ipfw show
00100      0         0 allow pfsync from any to any
00110      0         0 allow carp from any to any
00120      0         0 allow ip from any to any layer2 mac-type 0x0806,0x8035
00130      0         0 allow ip from any to any layer2 mac-type 0x888e,0x88c7
00140      0         0 allow ip from any to any layer2 mac-type 0x8863,0x8864
00150      0         0 deny ip from any to any layer2 not mac-type 0x0800,0x86dd
00200      0         0 skipto 60000 ip6 from ::1 to any
00201   2472   3379472 skipto 60000 ip4 from 127.0.0.0/8 to any
00202      0         0 skipto 60000 ip6 from any to ::1
00203      0         0 skipto 60000 ip4 from any to 127.0.0.0/8
06000 678786 579395407 skipto 60000 tcp from any to any out
06199 805279 667974243 skipto 60000 ip from any to any
30000      0         0 count ip from any to any
60000      0         0 return ip from any to any
60001  23016  12858000 queue 10003 esp from any to *ipsec site* via igb0 // wan: ipsec upload
60002  21873  19908104 queue 10002 esp from *ipsec site* to any via igb0 // wan: ipsec download
60003 402163 550469553 queue 10000 ip from any to *lan subnet* via igb0 // wan: download queue
60004 269819  31212155 queue 10001 ip from *lan subnet* to any via igb0 // wan: upload queue
65533 769666 636301310 allow ip from any to any
65534      0         0 deny ip from any to any
65535    106    102938 allow ip from any to any


4

General Discussion / Re: fq_codel latency spikes

« on: May 22, 2019, 07:50:28 pm »

I could certainly do that for VoIP but I will still need to resolve this issue with the pipe and scheduler. Latency should not be spiking when new downloads begin. It appears I'm exceeding the bandwidth set in my pipe by quite a lot; traffic graph shows 10.5-11 when my download begins with an 8 megabit bandwidth limit on my pipe. I'm not sure how this can be. Maybe my rules are incomplete? They seem very simple and air-tight to me, which leads me to believe that I'm missing some really foundational concept.

5

General Discussion / fq_codel latency spikes

« on: May 21, 2019, 05:31:59 pm »

I have a 10 megabit MetroE through Frontier and I'm sharing it among too many users. I've been using fq_codel to try to keep latency low for VoIP and gaming while providing reasonable bandwidth for streamers and downloads. I'm failing miserably.

I've set up fq_codel and it mostly works (I get an A+ for bufferbloat from dslreports) but when new streamers/downloads/tests start latency will momentarily spike by around 100 ms. This kills the gamers and happens quite often during peak time.

I've no idea what's wrong with my setup. Please help!

# ipfw sched show

Code: [Select]

10000:   8.000 Mbit/s    0 ms burst 0
q75536  50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail
 sched 10000 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 50ms quantum 1514 limit 10240 flows 1024 NoECN
   Children flowsets: 10000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0        4      669  0    0   0
10001:   8.000 Mbit/s    0 ms burst 0
q75537  50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail
 sched 10001 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 50ms quantum 1514 limit 10240 flows 1024 NoECN
   Children flowsets: 10001
  0 ip           0.0.0.0/0             0.0.0.0/0        3      457  0    0   0

# ipfw pipe show

Code: [Select]

10000:   8.000 Mbit/s    0 ms burst 0
q75536  50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail
 sched 75536 type FIFO flags 0x0 0 buckets 0 active
10001:   8.000 Mbit/s    0 ms burst 0
q75537  50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail
 sched 75537 type FIFO flags 0x0 0 buckets 0 active


Code: [Select]

60001   48276  17525072 queue 10001 esp from any to *ipsec site* via igb0 // wan: upload queue
60002   61714  76825600 queue 10000 esp from *ipsec site* to any via igb0 // wan: download queue
60003  426694 548488221 queue 10000 ip from any to *lan subnet* via igb0 // wan: download queue
60004  317134  43959271 queue 10001 ip from *lan subnet* to any via igb0 // wan: upload queue


6

General Discussion / Re: IPSec and fq_codel

« on: January 16, 2019, 08:31:41 pm »

Thanks so much, mimugmail! Setting protocol to esp did the trick.

7

General Discussion / Re: IPSec and fq_codel

« on: January 16, 2019, 07:50:29 pm »

I tried adding WAN rules:

from other ipsec host, dest port 500 to my download pipe
to other ipsec host, dest port 500 to my upload pipe

does not seem to work and I'm not sure why

8

General Discussion / IPSec and fq_codel

« on: January 16, 2019, 07:08:59 pm »

I've a small community of friends in a rural community sharing the same 10 megabit copper-over-Ethernet link. fq_codel was working fine to keep latency low and allow maximum available bandwidth for every user- really brilliant- but we also have a site-to-site IPSec bridge using this bridge to download/upload files seems to saturate the link creating really high latency and such for everyone. It doesn't appear to be hitting the queue at all. How can I manage this?