OPNsense Forum

Archive => 23.1 Legacy Series => Topic started by: rocketraman on April 19, 2023, 01:04:11 am

Title: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 19, 2023, 01:04:11 am
The automatically generated ICMP rules for IPV6 include this rule:

IPv6 IPV6-ICMP   fe80::/10   *   fe80::/10,ff02::/16   *   *   *   *   IPv6 RFC4890 requirements (ICMP)

as well as this rule further down the list:

IPv6 *   <bogonsv6>   *   *   *   *   *      Block bogon IPv6 networks from WAN

However, in my logs I see:

Apr 18 19:01:58 router.home.arpa filterlog[31353]: 71,,,f140a48ddade668b9d6f5259669a1d5c,igb0,match,block,in,6,0xe0,0x00000,1,icmp,1,32,fe80::10,ff02::1:ff00:1,truncated-ip6=32


The traffic is being blocked by the "Block bogon IPV6 networks from WAN" rule, but as far as I can tell from the source `fe80::10` and destination `ff02::1:ff00:1` this traffic should be passed by the "IPv6 RFC4890 requirements (ICMP)" rule. Why isn't it?

I have reset the state table to no effect.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 19, 2023, 01:27:29 am
Ok, I see from

pfctl -s rules | grep "from fe80::/10"

that these rules actually pass specific ICMP types, rather than all ICMP with this source and destination as the GUI appears to show. The type of this blocked traffic is "1", which is currently unassigned, so I guess it makes sense that the Bogon rule would trigger.

Is there any particular reason I would be seeing such bogons inbound on my WAN interface? Could it possibly be a misconfiguration of some sort on my part, or perhaps on my ISPs part?
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: nghappiness on April 19, 2023, 04:22:49 pm
ICMPv6 type 1 is destination unreachable.

ff02::1:ff00:1 is a solicited-node multicast address.

Which IPv6 device has the fe80::10 local address?   Can you login to opnsense cli and check ndp -a output against the arp -a output for your ISP IPv4 router mac address?   

See if the fe80::10 is coming from your ISP?

Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 19, 2023, 08:12:32 pm
ICMPv6 type 1 is destination unreachable.

Ah ok. Yup, I see that now.

ff02::1:ff00:1 is a solicited-node multicast address.

Which IPv6 device has the fe80::10 local address?   Can you login to opnsense cli and check ndp -a output against the arp -a output for your ISP IPv4 router mac address?   

Yes, `ndp` shows fe80::10 is MAC 00:5f:86:92:08:19, and igb0 is the WAN interface:

fe80::10%igb0                        00:5f:86:92:08:19   igb0 1m23s     R R

`arp` shows this is my ISPs router -- the IP address is the WAN ivp4 gateway.

See if the fe80::10 is coming from your ISP?

I guess it is. `ff02::1` IIUC is a broadcast to all nodes. It seems like this traffic is legitimate and should pass the firwewall?
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 19, 2023, 08:21:36 pm
If I uncheck "Block bogon networks" on Interfaces/WAN, I think some of this ISP traffic appears to be passed successfully -- I see far fewer blocks of this traffic in the logs. I do still see some blocked -- the live log shows the blocked traffic as "Default deny / state violation rule" trigger, but is that normal?

Apr 19 14:28:40 router.home.arpa filterlog[55817]: 11,,,02f4bab031b57d1e30553ce08e0ec131,igb0,match,block,in,6,0xe0,0x00000,1,icmp,1,32,fe80::10,ff02::1:ff00:1,truncated-ip6=32

Seems like a bug in the default firewall ruleset?
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: nghappiness on April 19, 2023, 08:45:00 pm
I checked my filter log, I don't see the same type of log as describe.

Here is a guess,

Some IPv6 devices on your network is trying to reach another IPv6 device on Internet.  But the ISP router is not able to reach that destination and sending you a destination unreachable? 

If you are seeing the same "src, dst, and ICMPv6 type 1" repeatly in the filter log.   That session never got aged out.  Regardless if you uncheck "Block bogon networks" and save.  You may want to try the following,

Firewall > Diagnostics > States and filter that session and hit the trash can command to clear that session?

or you may need to reset the state table.  Firewall > Diagnostics > States > Actions.  That will drop the entire session table.  Anything are transferring, they will need to re-establish new connecitons.

It will be interesting to figure out what is causing the ICMPv6 destination unreachable, can you stop all IPv6 traffic from your lans/trust to wan/Internet and see if that stops those filter logs? 
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 19, 2023, 10:03:10 pm
Ok, seems I may have misunderstood something important... the `icmp,1` in the log is not ICMP type 1 but rather protocol 1 (which just means ICMP again).

I couldn't see a way to determine the ICMP type from the logs so I captured the traffic with tcpdump, and see that the type is "Multicast Listener Report". The content of the packet seems pretty innocuous:

Internet Control Message Protocol v6
    Type: Multicast Listener Report (131)
    Code: 0
    Checksum: 0x8210 [correct]
    [Checksum Status: Good]
    Maximum Response Delay [ms]: 0
    Reserved: 0000
    Multicast Address: ff02::1:ff00:1


I see packets of type "Multicast Listener Report" from fe80::10 to several destinations, including ff02::1:2, ff02::5, ff02::6, as well as the solicited-node multicast address ff02::1:ff00:1. They all have the same (save for the checksum and multicast address) packet content.

I also blocked all outbound ipv6 traffic and reset the state table, and still saw these packets inbound. So as far as I can tell this is "normal" traffic from my ISP, but opnsense is blocking it.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: nghappiness on April 19, 2023, 11:09:23 pm
Quote
Ok, seems I may have misunderstood something important... the `icmp,1` in the log is not ICMP type 1 but rather protocol 1 (which just means ICMP again).

The protcol is ICMPv6, not ICMP. ICMPv6 protocol number is 58. 

Quote
I also blocked all outbound ipv6 traffic and reset the state table, and still saw these packets inbound. So as far as I can tell this is "normal" traffic from my ISP, but opnsense is blocking it.

Time for a ticket to the provider?  or they are supporting multicast? 
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 20, 2023, 09:11:26 am
I stilll don't understand something fundamental here... the packet is blocked by the "Block bogon IPv6 networks from WAN" rule, but fe80::10 isn't in the bogon list. Furthermore the packet protocol is indeed logged as 1 (ICMP), not 58 (ICMPv6). Something very strange about this packet being blocked in this way that I don't grok at all.

I can only imagine having this discussion with my provider :-)
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: nghappiness on April 20, 2023, 01:31:43 pm
I am interested to take a closely look at the matching sample pcap and filter log, if possible.    What version of opnsense are you running?  the filter log you posted before looks different than the one on my firewall.

I found this in my filter log.

1 2023-04-07T20:16:46-07:00 opn.home.net filterlog 16947 - [meta sequenceId="151715"] 131,,,b8350fa47ada7fe6c07ace7650fc4dcc,vlan01,match,pass,in,6,0x00,0x00000,1,icmp,1,36,fe80::<redux>,ff02::16,truncated-ip6=36

I wonder if it is because truncated-ipv6 , the filter log just print out the protocol name incorrectly..

See if you can find the rule matched under Firewall: Log files: Live View and apply filter like interface contains igb0 and action is block?

Just need to make sure those automatically generated rules are logged.  (system > settings > Logging > Log packets matched from the default pass rules put in the ruleset is checked. )
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 20, 2023, 04:12:17 pm
I am interested to take a closely look at the matching sample pcap and filter log, if possible.    What version of opnsense are you running?  the filter log you posted before looks different than the one on my firewall.

I found this in my filter log.

1 2023-04-07T20:16:46-07:00 opn.home.net filterlog 16947 - [meta sequenceId="151715"] 131,,,b8350fa47ada7fe6c07ace7650fc4dcc,vlan01,match,pass,in,6,0x00,0x00000,1,icmp,1,36,fe80::<redux>,ff02::16,truncated-ip6=36

I wonder if it is because truncated-ipv6 , the filter log just print out the protocol name incorrectly..

That seems like a reasonable supposition. Here is the version info:

OPNsense 23.1.5_4-amd64
FreeBSD 13.1-RELEASE-p7
OpenSSL 1.1.1t 7 Feb 2023


See if you can find the rule matched under Firewall: Log files: Live View and apply filter like interface contains igb0 and action is block?

Just need to make sure those automatically generated rules are logged.  (system > settings > Logging > Log packets matched from the default pass rules put in the ruleset is checked. )

Yes, the packets are visible in the Live View, here is a screenshot of the live view:

https://i.imgur.com/PIcWokD.png

And here is the detailed info from the live view for one of the blocked packets:

https://i.imgur.com/rpAgqrf.png

I have attached a packet capture on the wan interface for 'host fe80::10' at the same time as the screenshot.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: nghappiness on April 20, 2023, 09:50:23 pm
I found that automatic floating rule under my wan interface. 

Can you go to Firewall > Aliases  and look for bogonsv6 , what is the number in the loaded column.  I have 140163.

Login to CLI, go to /tmp/bogons

drwxr-x---  2 root  wheel        6 Apr 16 03:13 .
drwxrwxrwt  8 root  wheel       40 Apr 20 12:52 ..
-rw-r-----  1 root  wheel    55232 Apr 12 01:15 bogons.txz
-rw-r-----  1 root  wheel     1332 Apr 12 01:15 bogons.txz.sig
-rw-r--r--  1 root  wheel    14910 Apr 11 21:55 fullbogons-ipv4.txt
-rw-r--r--  1 root  wheel  2306406 Apr 11 21:55 fullbogons-ipv6.txt

I got nothing begin with fe80 in fullbogons-ipv6.txt.

I also checked on Firewall > Diagnostics > Aliases > selected bogonsv6 from the drop down (top left).  I don't see any entries begin with fe80.

Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: IsaacFL on April 20, 2023, 10:37:43 pm
The Bogons list includes 8000::/1  I think that includes the FE80

So yes block bogons will block link local.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: nghappiness on April 20, 2023, 11:46:23 pm
Hi IsaacFL,

Yes, thanks for pointing it out!  I need to get a pencil and paper to do some binary math.. 
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: IsaacFL on April 21, 2023, 12:04:50 am
I finally decided that you cannot use the Block Bogons for ipv6 as it breaks the protocol.

What I did was create an alias to download the bogons list directly from http://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt

Then I created an alias for exclusions for !fe80::/10, !ff00::/8, !::1, and since I use NAT64 also !64:ff9b::/96.  I also use ULA internally, so I have an exclusion for the ULA prefix I am using too.

Then I created a another alias for bogons with the exclusions.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 21, 2023, 09:54:35 pm
The Bogons list includes 8000::/1  I think that includes the FE80

So yes block bogons will block link local.

Oh, you're right! 8000::/1 includes everything 8000 and above! Ipv4 has conditioned me to never consider anything less than a /8 on the CIDR.

At least now I know I'm not crazy -- but this seems like a questionable decision on the bogon list. Does anyone know if this has been raised an issue anywhere else?
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 21, 2023, 10:34:38 pm
I finally decided that you cannot use the Block Bogons for ipv6 as it breaks the protocol.

What I did was create an alias to download the bogons list directly from http://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt

Then I created an alias for exclusions for !fe80::/10, !ff00::/8, !::1, and since I use NAT64 also !64:ff9b::/96.  I also use ULA internally, so I have an exclusion for the ULA prefix I am using too.

Then I created a another alias for bogons with the exclusions.

Why are you excluding ::1?
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: Patrick M. Hausen on April 22, 2023, 12:30:34 am
Coming in from any real interface? Why of course. It's not a valid source address on any wire.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: rocketraman on April 22, 2023, 12:47:26 am
Coming in from any real interface? Why of course. It's not a valid source address on any wire.

Exactly. In this context, excluding `::1` from the bogon list means allowing it i.e. not considering it a bogon. Hence my question.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: IsaacFL on April 22, 2023, 12:51:15 am
I finally decided that you cannot use the Block Bogons for ipv6 as it breaks the protocol.

What I did was create an alias to download the bogons list directly from http://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt

Then I created an alias for exclusions for !fe80::/10, !ff00::/8, !::1, and since I use NAT64 also !64:ff9b::/96.  I also use ULA internally, so I have an exclusion for the ULA prefix I am using too.

Then I created a another alias for bogons with the exclusions.

Why are you excluding ::1?

I don’t remember but it must have been showing in firewall logs under my bogon block rule so I excluded it?
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: IsaacFL on April 22, 2023, 12:53:32 am
Coming in from any real interface? Why of course. It's not a valid source address on any wire.

Exactly. In this context, excluding `::1` from the bogon list means allowing it i.e. not considering it a bogon. Hence my question.

The default deny rule would catch it, but I am questioning it myself now.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: IsaacFL on April 22, 2023, 01:03:16 am
The Bogons list includes 8000::/1  I think that includes the FE80

So yes block bogons will block link local.

Oh, you're right! 8000::/1 includes everything 8000 and above! Ipv4 has conditioned me to never consider anything less than a /8 on the CIDR.

At least now I know I'm not crazy -- but this seems like a questionable decision on the bogon list. Does anyone know if this has been raised an issue anywhere else?

I question it myself cause it since it is in the autogenerated rules you can’t override it.  I especially don’t understand the ULA prefix because they are technically allowed to be routed at least locally. Site level Multicast prefix maybe also, though opnsense doesn’t support that.
Title: Re: Automatically generated ICMP rules for IPV6 not passing taffic
Post by: Reiner030 on June 02, 2023, 04:38:50 pm
Hi,
I found this thread while I have similar problems to cleanup our firewall log files from wrongly blocked packets (preparing migration from pfSense to OPNsense business "OPNsense 23.4-amd64").

We are using
Problem for both events is that these log lines:
Code: [Select]
WAN108_Uplink_VDSL_Telekom 2023-06-02T16:05:05 fe80::1 ff02::1 ip Block bogon IPv6 networks from WAN108_Uplink_VDSL_Telekom
WAN110_Uplink_Cable_KD 2023-06-02T16:04:04 [fe80::ca0e:14ff:fe6c:4bcc]:53805 [ff02::1]:53805 udp Block bogon IPv6 networks from WAN110_Uplink_Cable_KD
can only be deactivated by deactivating "Block bogon networks" additional to "Block private networks" on their interfaces.
Is there some special place for this request? So far I can see this forum seems the best one for my verrsion.