Networking/PF : strange behaviour with ICMPv6 pf matching

[pfoo]

Hi !

I think I need some help and community insight.

SW/HW :

Code Select Expand

OPNsense 25.1.5_4-amd64
FreeBSD 14.2-RELEASE-p2
OpenSSL 3.0.16
Hardware : DEC750

Background : i'd need to tag outbound ICMPv6 RA/NS/NA to 802.1Q priority 6 on an WAN interface assigned with a vlan.

For now there is no way to make opnsense match those packets due to these hardcoded automatic rules :

Code Select Expand

pass out log quick inet6 proto ipv6-icmp from {(self)} to {fe80::/10} icmp6-type {128,129,133,134,135,136} keep state label "247d6ba2cf9b0caa4e483f8f98f7a480" # IPv6 RFC4890 requirements (ICMP)
pass out log quick inet6 proto ipv6-icmp from {(self)} to {ff02::/16} icmp6-type {128,129,133,134,135,136} keep state label "247d6ba2cf9b0caa4e483f8f98f7a480" # IPv6 RFC4890 requirements (ICMP)That's for a another discussion as I think that adding user defined rule before hardcoded/automatic rules was already discussed many times.

For test purpose I manually added rules (before the two default rules I stated above) in rules.debug and reloaded pf (pfctl -f /tmp/rules.debug) and monitored packets.

Sidenote : in this configuration, the router is strangely generating 2 icmp6-router solicitation message each time : one on vlan832 and one with no vlan (why ?)

Back to the main issue.
Adding simple rules like this :

Code Select Expand

pass out log quick inet6 proto ipv6-icmp from {(self)} to {fe80::/10} icmp6-type {133,135,136} keep state set prio 6 label "xyz" # test
pass out log quick inet6 proto ipv6-icmp from {(self)} to {ff02::/16} icmp6-type {133,135,136} keep state set prio 6 label "xyz" # testMakes icmp6 type 133 (at least, didn't try for 135/136) correctly tagged with priority 6.
This rule will however have side effects as it will tag (and set an empty vlan tag with vlanid=0) for icmp6 133/135/136 leaving the firewall by any interface and not only WAN. This can interfere with LAN switches configured with some strict vlan filtering so I'd prefer to avoid that.

However if I add the WAN vlan interface (vlan0.1.832) in the rule : no priority added anymore :

Code Select Expand

pass out log quick on vlan0.1.832 inet6 proto ipv6-icmp from {(self)} to {fe80::/10} icmp6-type {133,135,136} keep state set prio 6 label "xyz" # test
pass out log quick on vlan0.1.832 inet6 proto ipv6-icmp from {(self)} to {ff02::/16} icmp6-type {133,135,136} keep state set prio 6 label "xyz" # test-> no priority added to packets
I tried removing from / to statement, no more luck.

If I add the vlan parent interface (igc0) in the rule the priority is added again :

Code Select Expand

pass out log quick on igc0 inet6 proto ipv6-icmp from {(self)} to {fe80::/10} icmp6-type {133,135,136} keep state set prio 6 label "xyz" # test
pass out log quick on igc0 inet6 proto ipv6-icmp from {(self)} to {ff02::/16} icmp6-type {133,135,136} keep state set prio 6 label "xyz" # test-> priority added

At this point I'm quite not understanding why specifying the vlan interface breaks the rule, as using dhcpvlanprio and dhcp6vlanprio in order to set priority of dhcp/dhcp6 packets is working flawlessly with similar autogenerated rules (but for udp instead of icmp), eg from rules.debug :

Code Select Expand

pass out log quick on vlan0.1.832 proto udp from {fe80::/10} port {546} to {fe80::/10} port {547} set prio 6 label "d3fb7cbab7078ccf431c3101dfeb4a87" # allow dhcpv6 client out WAN
pass out log quick on vlan0.1.832 proto udp from {fe80::/10} port {546} to {ff02::/16} port {547} set prio 6 label "d3fb7cbab7078ccf431c3101dfeb4a87" # allow dhcpv6 client out WAN
I could stick to setting priority by matching interface igc0 in pf ... however if I try to patch /usr/local/etc/inc/filter.lib.inc in order to add a rule matching igc0 I end with a debug error :

#debug:Interface igc0 not found

This can be fixed by adding and enabling the vlan parent interface igc0 as opt1 in opnsense webui to pass rule validation (is there some side effects of mixing enabled non-vlan and vlan interface ?)
However, if I add and enable the vlan parent interface :
- my pf rule matching on interface igc0 is not working anymore
- my pf rule matching on interface vlan0.1.832 is magically working

Sidenote : in this configuration (parent and vlan interface enabled, no ip assigned to parent interface), the router is generating only 1 icmp6-RS (for vlan832)

If someone has any idea on what's preventing my pf rule to match packet on vlan interface without enabling the parent interface ..

[pfoo]

Ok, two coffee later I might have found something :
- removed parent interface from opnsense webui
- reboot to assume clean state
- disable ACCEPT_RTADV flag from igc0 : ifconfig igc0 inet6 -accept_rtadv

-> only one icmp6-RouterSolicitation is generated, on vlan832, and is correctly tagged with priority 6 by my pf rule (on the vlan interface).

If I remember correctly some time ago an opnsense change forced to enable vlan parent interface. This change was quickly reversed and parent interface were autoconfigured (https://docs.opnsense.org/releases/CE_22.7.html#september-07-2022)

At least with a VLAN setup, having ACCEPT_RTADV flag on an unconfigured/unassigned vlan parent interface generate ICMP6-RouterSolicitation messages and probably others ND6 messages (this feels wrong) with no vlanid while another ICMP6-RouterSolicitation message is generated with the correct vlanid. This, in some ways, seems to break pf from matching icmp6 messages on the vlan interface (why?).

I have not enough network knowledge and about no freebsd knowledge but this clearly have some implications so : is it safe to remove ACCEPT_RTADV flag on vlan parent interface ?

Should I consider this a bug and open a ticket ?