Gateway Policy being ignored for IPv6 route

[bluknight]

So I have 2 IPv6 uplinks - aaaa:bbbb:cccc:dddd::/60 from my ISP, and eeee:ffff:gggg::/48 from TunnelBroker.  The /48 feeds my lab, and other services I run from my house that I don't want my ISP to twiddle with.  I'm trying to do a software update on a server within the /48 (no, not the firewall), pulling from repositories at wwww:xxxx::yyyy:zzzz.

I've run into a problem where if I'm connecting outbound from a server in the /48, the traffic goes out my ISP uplink (let's say qqqq:rrrr:ssss:tttt::1) instead of across the tunnel (a gateway of hhhh:iiii:jjjj:kkkk::1).

I've put (sanitized) outputs from my filter.log below, plus the corresponding rules from pfctl -vvsr.  I'm new to pf and OpnSense (I'm more of a Linux iptables/netfilter guy) but I've gotta learn sometime, and I can only assume I'm doing something monumentally stupid.

Help a fellow hacker who's down on his luck?

Code: [Select]

Mar 22 21:16:26 OPNsense filterlog: 159,,,0,re0_vlan3000,match,pass,in,6,0x00,0xc1f1e,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34912,80,0,S,3326026852,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:16:26 OPNsense filterlog: 130,,,0,gif0,match,pass,out,6,0x00,0xc1f1e,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34912,80,0,S,3326026852,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:20:26 OPNsense filterlog: 159,,,0,re0_vlan3000,match,pass,in,6,0x00,0xcf121,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34948,80,0,S,1374304364,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:20:26 OPNsense filterlog: 130,,,0,gif0,match,pass,out,6,0x00,0xcf121,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34948,80,0,S,1374304364,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:22:58 OPNsense filterlog: 159,,,0,re0_vlan3000,match,pass,in,6,0x00,0xfaa0c,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34976,80,0,S,2639873281,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:22:58 OPNsense filterlog: 130,,,0,gif0,match,pass,out,6,0x00,0xfaa0c,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34976,80,0,S,2639873281,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:22:58 OPNsense filterlog: 159,,,0,re0_vlan3000,match,pass,in,6,0x00,0x3f3c9,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34978,80,0,S,1042769232,,28800,,mss;sackOK;TS;nop;wscale
Mar 22 21:22:58 OPNsense filterlog: 130,,,0,gif0,match,pass,out,6,0x00,0x3f3c9,64,tcp,6,40,eeee:ffff:gggg:0:20c:29ff:fee2:f444,wwww:xxxx::yyyy:zzzz,34978,80,0,S,1042769232,,28800,,mss;sackOK;TS;nop;wscale


Code: [Select]

@130 pass out log all flags S/SA keep state allow-opts label "a5a4a52cb247a9d532d9e49588136184"

@159 pass in log quick on re0_vlan3000 route-to (gif0 hhhh:iiii:jjjj:kkkk::1) inet6 from eeee:ffff:gggg::/64 to ! aaaa:bbbb:cccc:dddd::/60 flags S/SA keep state label "e4344ac62537cc4ca99bf17147e9ce69"


[mimugmail]

Firewall : Settings : Advanced .. can you try disabling shared forwaring?

[bluknight]

Firewall : Settings : Advanced .. can you try disabling shared forwaring?

That nailed it -- traffic outbound is working.  Re-enabling broke it, so that validated the fix.  Thank you for the help!

Now...  can you help me understand why that fixed it?

[franco]

Internal workings of the IPv6 stack parts are different from IPv4 which renders shared forwarding useless. Shared forwarding tries to stop pf(4) from hiding packets from the rest of the network stack.

So if you disable shared forwarding pf(4) will immediately send them but the traffic shaper, captive portal or web proxy will not be able to see them.

pf(4) is doing the gateway policy enforcement either way.


Cheers,
Franco