Messages

My WAN link is down due to a fibre cut.

One of my LAN boxes (a Linux box) is acting as a temporary gateway to the WAN via a WiFi connection and mobile hotspot.

What is the best way to configure OPNSense to route WAN traffic to this particular Linux box, which can forward the traffic to the WIFI interface?

I believe it'll be a gateway configuration, but some specific advice on how to do it would be great. I already lost local connectivity once to opnsense with a misconfiguration (thanks for the ability to restore!).

Thank you for that, I've updated to 23.7.1 now!

That is a well-known problem that was adressed by 23.7.

Oh, thank you. I searched in the release notes for radvd but didn't find anything. I guess this item is probably the relevant one:

interfaces: on forceful IPv6 reload do not lose the event handling

My router uses "Track Interface" for LAN IPv6 Configuration Type, Manual Configuration checked. DHCPv6 is disabled, and Router Advertisements are "Unmanaged".

I had a network blip of a few seconds and when the network came back up, the router did have a WAN IPv6 address assigned but my LAN clients lost their ipv6 addresses. Restarting the radvd service manually restored IPv6 to LAN clients.

Looking at the logs I see these radv and network related logs when the network goes down:

Code Select Expand

Aug 12 01:37:27 router.home.arpa radvd[44978]: exiting, 1 sigterm(s) received
Aug 12 01:37:27 router.home.arpa radvd[44978]: sending stop adverts
Aug 12 01:37:27 router.home.arpa radvd[44978]: removing /var/run/radvd.pid
Aug 12 01:37:27 router.home.arpa radvd[44978]: returning from radvd main
Aug 12 01:37:28 router.home.arpa dhcp6c[57660]: transmit failed: Network is down
Aug 12 01:37:29 router.home.arpa dhcp6c[57660]: transmit failed: Network is down
Aug 12 01:37:31 router.home.arpa dhcp6c[57660]: transmit failed: Network is down

and then these logs whe the network comes back up a few seconds later:

Code Select Expand

Aug 12 01:37:35 router.home.arpa opnsense[99737]: /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for wan(igb0)
Aug 12 01:37:35 router.home.arpa dhcp6c[15717]: RTSOLD script - Sending SIGHUP to dhcp6c
Aug 12 01:37:35 router.home.arpa opnsense[...]: [...snip bunch of etc/rc.linkup etc/rc.newwanip stuff...]
Aug 12 01:37:36 router.home.arpa opnsense[99737]: /usr/local/etc/rc.linkup: dhcpd_radvd_configure(manual) found no suitable IPv6 address on igb3
Aug 12 01:37:36 router.home.arpa radvd[33395]: version 2.19 started
Aug 12 01:37:36 router.home.arpa dhcp6c[40844]: RTSOLD script - Sending SIGHUP to dhcp6c
Aug 12 01:37:38 router.home.arpa dhcp6c[71649]: dhcp6c_script: REQUEST on igb0 executing
Aug 12 01:37:38 router.home.arpa dhcp6c[73996]: dhcp6c_script: REQUEST on igb0 renewal
Aug 12 01:37:38 router.home.arpa opnsense[74889]: /usr/local/etc/rc.newwanipv6: Failed to detect IP for WAN[wan]
Aug 12 01:37:35 router.home.arpa opnsense[...]: [...snip bunch of etc/rc.linkup etc/rc.newwanip stuff...]

and then when I fixed things by restarting the radvd service:

Code Select Expand

Aug 12 13:37:17 router.home.arpa dhcp6c[94122]: dhcp6c_script: RENEW on igb0 executing
Aug 12 13:37:17 router.home.arpa dhcp6c[96448]: dhcp6c_script: RENEW on igb0 executing
Aug 12 13:37:17 router.home.arpa opnsense[95010]: /usr/local/etc/rc.newwanipv6: No IP change detected (current: 2001:1970:4000:a2::xxxx, interface: WAN[wan])
Aug 12 13:37:17 router.home.arpa opnsense[177]: /usr/local/etc/rc.newwanipv6: No IP change detected (current: 2001:1970:4000:a2::xxxx, interface: WAN[wan])


I note there are no messages from `etc/rc.newwanipv6` in between the message "Failed to detect IP for WAN[wan] at Aug 12 01:37:38 and the message at Aug 12 13:37:17 "No IP change detected (current: 2001:1970:4000:a2::xxxx, interface: WAN[wan])". This makes me think there is some kind of timing issue or missing event when the WAN ipv6 address is assigned.

Coming in from any real interface? Why of course. It's not a valid source address on any wire.

Exactly. In this context, excluding `::1` from the bogon list means allowing it i.e. not considering it a bogon. Hence my question.

I finally decided that you cannot use the Block Bogons for ipv6 as it breaks the protocol.

What I did was create an alias to download the bogons list directly from http://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt

Then I created an alias for exclusions for !fe80::/10, !ff00::/8, !::1, and since I use NAT64 also !64:ff9b::/96.  I also use ULA internally, so I have an exclusion for the ULA prefix I am using too.

Then I created a another alias for bogons with the exclusions.

Why are you excluding ::1?

The Bogons list includes 8000::/1  I think that includes the FE80

So yes block bogons will block link local.

Oh, you're right! 8000::/1 includes everything 8000 and above! Ipv4 has conditioned me to never consider anything less than a /8 on the CIDR.

At least now I know I'm not crazy -- but this seems like a questionable decision on the bogon list. Does anyone know if this has been raised an issue anywhere else?

I am interested to take a closely look at the matching sample pcap and filter log, if possible.    What version of opnsense are you running?  the filter log you posted before looks different than the one on my firewall.

I found this in my filter log.

1 2023-04-07T20:16:46-07:00 opn.home.net filterlog 16947 - [meta sequenceId="151715"] 131,,,b8350fa47ada7fe6c07ace7650fc4dcc,vlan01,match,pass,in,6,0x00,0x00000,1,icmp,1,36,fe80::<redux>,ff02::16,truncated-ip6=36

I wonder if it is because truncated-ipv6 , the filter log just print out the protocol name incorrectly..

That seems like a reasonable supposition. Here is the version info:

OPNsense 23.1.5_4-amd64
FreeBSD 13.1-RELEASE-p7
OpenSSL 1.1.1t 7 Feb 2023

See if you can find the rule matched under Firewall: Log files: Live View and apply filter like interface contains igb0 and action is block?

Just need to make sure those automatically generated rules are logged.  (system > settings > Logging > Log packets matched from the default pass rules put in the ruleset is checked. )

Yes, the packets are visible in the Live View, here is a screenshot of the live view:

https://i.imgur.com/PIcWokD.png

And here is the detailed info from the live view for one of the blocked packets:

https://i.imgur.com/rpAgqrf.png

I have attached a packet capture on the wan interface for 'host fe80::10' at the same time as the screenshot.

I stilll don't understand something fundamental here... the packet is blocked by the "Block bogon IPv6 networks from WAN" rule, but fe80::10 isn't in the bogon list. Furthermore the packet protocol is indeed logged as 1 (ICMP), not 58 (ICMPv6). Something very strange about this packet being blocked in this way that I don't grok at all.

I can only imagine having this discussion with my provider :-)

[type]

Ok, seems I may have misunderstood something important... the `icmp,1` in the log is not ICMP type 1 but rather protocol 1 (which just means ICMP again).

I couldn't see a way to determine the ICMP type from the logs so I captured the traffic with tcpdump, and see that the type is "Multicast Listener Report". The content of the packet seems pretty innocuous:

Internet Control Message Protocol v6
    Type: Multicast Listener Report (131)
    Code: 0
    Checksum: 0x8210 [correct]
    [Checksum Status: Good]
    Maximum Response Delay [ms]: 0
    Reserved: 0000
    Multicast Address: ff02::1:ff00:1

I see packets of type "Multicast Listener Report" from fe80::10 to several destinations, including ff02::1:2, ff02::5, ff02::6, as well as the solicited-node multicast address ff02::1:ff00:1. They all have the same (save for the checksum and multicast address) packet content.

I also blocked all outbound ipv6 traffic and reset the state table, and still saw these packets inbound. So as far as I can tell this is "normal" traffic from my ISP, but opnsense is blocking it.

[some]

If I uncheck "Block bogon networks" on Interfaces/WAN, I think some of this ISP traffic appears to be passed successfully -- I see far fewer blocks of this traffic in the logs. I do still see some blocked -- the live log shows the blocked traffic as "Default deny / state violation rule" trigger, but is that normal?

Apr 19 14:28:40 router.home.arpa filterlog[55817]: 11,,,02f4bab031b57d1e30553ce08e0ec131,igb0,match,block,in,6,0xe0,0x00000,1,icmp,1,32,fe80::10,ff02::1:ff00:1,truncated-ip6=32

Seems like a bug in the default firewall ruleset?

ICMPv6 type 1 is destination unreachable.

Ah ok. Yup, I see that now.

ff02::1:ff00:1 is a solicited-node multicast address.

Which IPv6 device has the fe80::10 local address?   Can you login to opnsense cli and check ndp -a output against the arp -a output for your ISP IPv4 router mac address?   

Yes, `ndp` shows fe80::10 is MAC 00:5f:86:92:08:19, and igb0 is the WAN interface:

fe80::10%igb0                        00:5f:86:92:08:19   igb0 1m23s     R R

`arp` shows this is my ISPs router -- the IP address is the WAN ivp4 gateway.

See if the fe80::10 is coming from your ISP?

I guess it is. `ff02::1` IIUC is a broadcast to all nodes. It seems like this traffic is legitimate and should pass the firwewall?

Figured out one way to do it:

cat $(ifctl -6 -l -p)

The web interface shows the "IPv6 delegated prefix" value in the Interfaces/Overview under the WAN interface. How can I obtain this information from a shell?

The `ifconfig` command shows the delegated prefix under my LAN interface but with the wrong prefix length: /64 instead of /56 which shows on the web interface.

Ok, I see from

pfctl -s rules | grep "from fe80::/10"

that these rules actually pass specific ICMP types, rather than all ICMP with this source and destination as the GUI appears to show. The type of this blocked traffic is "1", which is currently unassigned, so I guess it makes sense that the Bogon rule would trigger.

Is there any particular reason I would be seeing such bogons inbound on my WAN interface? Could it possibly be a misconfiguration of some sort on my part, or perhaps on my ISPs part?