Multi-WAN, Policy Routing, and Traffic Shaping

[namezero111111]

Dear folks, in the past there seem to have been problems with mixing Shaping into Gateway redirect rules.
(see https://github.com/opnsense/core/issues/1230 ).

We are testing a deployment on 18.1.2 amd64, and are running into a similar problem:

1. A gateway group has been defined (2 GW's on different interfaces, A and B)
2. A firewall rule has been created to redirect specific traffic via the gateway group
3. Manual NAT is enabled
4. Traffic shaping for the two interfaces (A and B) is created
   - A pipe each with WFQ w/ codel
   - A "catch-all" default rule redirecting traffic into the pipe
5. Advanced->" Shared forwarding " is enabled (ticked)

Expected outcome: Traffic is limited to the pipe's bandwidth

Actual outcome: No limiting is applied, and the queue shows no connection/limiting in the status section.

Is there anything obvious missing?
In other setups the shaping (without Multi-WAN) works alright.

Thanks in advance!

Edit: I would like to add that no rules under "ipfw show" redirect anything into pipes or queues are showing up, even after reset and apply on the traffic shaper:

00100      3      2002 allow pfsync from any to any
00110     21      1176 allow carp from any to any
00120      0         0 allow ip from any to any layer2 mac-type 0x0806,0x8035
00130      0         0 allow ip from any to any layer2 mac-type 0x888e,0x88c7
00140      0         0 allow ip from any to any layer2 mac-type 0x8863,0x8864
00150      0         0 deny ip from any to any layer2 not mac-type 0x0800,0x86dd
00200      0         0 skipto 60000 ip6 from ::1 to any
00201      0         0 skipto 60000 ip4 from 127.0.0.0/8 to any
00202      0         0 skipto 60000 ip6 from any to ::1
00203      0         0 skipto 60000 ip4 from any to 127.0.0.0/8
01001      0         0 skipto 60000 udp from any to 192.168.4.6 dst-port 53 keep-state :default
01001     21      2114 skipto 60000 ip from any to { 255.255.255.255 or 192.168.4.6 } in
01001     28      8526 skipto 60000 ip from { 255.255.255.255 or 192.168.4.6 } to any out
01001      0         0 skipto 60000 icmp from { 255.255.255.255 or 192.168.4.6 } to any out icmptypes 0
01001      0         0 skipto 60000 icmp from any to { 255.255.255.255 or 192.168.4.6 } in icmptypes 8
01002      0         0 skipto 60000 udp from any to 192.168.4.77 dst-port 53 keep-state :default
01002      0         0 skipto 60000 ip from any to { 255.255.255.255 or 192.168.4.77 } in
01002      0         0 skipto 60000 ip from { 255.255.255.255 or 192.168.4.77 } to any out
01002      0         0 skipto 60000 icmp from { 255.255.255.255 or 192.168.4.77 } to any out icmptypes 0
01002      0         0 skipto 60000 icmp from any to { 255.255.255.255 or 192.168.4.77 } in icmptypes 8
01003      0         0 skipto 60000 udp from any to 192.168.4.131 dst-port 53 keep-state :default
01003      0         0 skipto 60000 ip from any to { 255.255.255.255 or 192.168.4.131 } in
01003      0         0 skipto 60000 ip from { 255.255.255.255 or 192.168.4.131 } to any out
01003      0         0 skipto 60000 icmp from { 255.255.255.255 or 192.168.4.131 } to any out icmptypes 0
01003      0         0 skipto 60000 icmp from any to { 255.255.255.255 or 192.168.4.131 } in icmptypes 8
01004      0         0 skipto 60000 udp from any to 192.168.4.146 dst-port 53 keep-state :default
01004      0         0 skipto 60000 ip from any to { 255.255.255.255 or 192.168.4.146 } in
01004      0         0 skipto 60000 ip from { 255.255.255.255 or 192.168.4.146 } to any out
01004      0         0 skipto 60000 icmp from { 255.255.255.255 or 192.168.4.146 } to any out icmptypes 0
01004      0         0 skipto 60000 icmp from any to { 255.255.255.255 or 192.168.4.146 } in icmptypes 8
06000      0         0 skipto 60000 tcp from any to any out
06199      0         0 skipto 60000 ip from any to any
30000      0         0 count ip from any to any
60000      0         0 return ip from any to any
65535 682972 605826657 allow ip from any to any

However, the queues are defined, as shown by ipfw queues show:

q10006  50 sl. 0 flows (1 buckets) sched 10000 weight 40 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10007  50 sl. 0 flows (1 buckets) sched 10001 weight 40 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10004  50 sl. 0 flows (1 buckets) sched 10001 weight 70 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10005  50 sl. 0 flows (1 buckets) sched 10002 weight 70 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10002  50 sl. 0 flows (1 buckets) sched 10002 weight 95 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10003  50 sl. 0 flows (1 buckets) sched 10002 weight 80 lmax 0 pri 0  AQM CoDel target 5ms interval 500ms NoECN
q10000  50 sl. 0 flows (1 buckets) sched 10000 weight 95 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10001  50 sl. 0 flows (1 buckets) sched 10001 weight 95 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10014  50 sl. 0 flows (1 buckets) sched 10002 weight 20 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10015  50 sl. 0 flows (1 buckets) sched 10002 weight 55 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10010  50 sl. 0 flows (1 buckets) sched 10002 weight 65 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10008  50 sl. 0 flows (1 buckets) sched 10002 weight 40 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10009  50 sl. 0 flows (1 buckets) sched 10001 weight 65 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q10016  50 sl. 0 flows (1 buckets) sched 10001 weight 55 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms NoECN

[namezero111111]

Inspired by another post I looked at the rules generated in /usr/local/etc/ipfw.rules and found that using a colon separator for port ranges in the shaper config makes ipfw choke.

The web interface allows bad input here, the colon should be automatically replaced with a dash.

I'n not sure if core/ipfw.conf is the correct place to fix this or whether earlier, in the GUI this should be rejected or both.