IPv6 link-local route does not survive reboot

[matt335672]

Hi,

I've got a couple of OpnSense routers running 25.7.11_2 linked together.

My primary router receives a static /48 from my ISP, and then delegates a /60 to the secondary router. This is all done with static routing rather than PD, as Kea can't currently add the route for a delegated prefix automatically - see https://forum.opnsense.org/index.php?topic=48680

The interface vtnet3 on the main router has an address of FE80::1/64 and the interface em0 on the secondary router has an address of FE80::2/64.

On the main router, I've got a gateway and a static route configured to reach the secondary router as follows (links to postimg.cc):



The secondary router just has FE80::1 on the main router set as a default gateway.

There's also an IPv4 config set up in  similar way.

This all works fine until I reboot the main router. When it comes up, the route for <prefix48>fff0::/60 to FE80::2%vtnet3 is missing from the routing table until I click the 'Apply' button on the routing config screen on the main router.

IPv4 works perfectly, although this is not using link-local addresses.

Any idea what I'm doing wrong here?

[Maurice]

Nothing wrong with your configuration, it's most likely a bug. I've seen this behaviour before under hard to reproduce circumstances - some static IPv6 routes sometimes don't get added to the routing table after a reboot. Haven't been able to pinpoint it and mostly worked around it with Monit.

If it's reproducible in your setup, creating an issue on GitHub with as many details as possible (logs) might be the best way forward.

Cheers
Maurice

[Monviech (Cedrik)]

Kea sets automatic routes now, it was a roadmap item:

https://docs.opnsense.org/manual/kea.html#prefix-delegation-ia-pd

[matt335672]

Thanks both.

I was using PD with ISC dhcpd for this, and it was working (mostly) fine. For what I'm doing there's no advantage in using PD really, so I'd like to get the static routes working.

I'm not keen on messing around with my primary router for this, so I'll try setting up a VM with just my static routes on it and see what happens. There are only 3. If I can reproduce it on that, it should make a fault report easier.

[franco]

> When it comes up, the route for <prefix48>fff0::/60 to FE80::2%vtnet3 is missing from the routing table until I click the 'Apply' button on the routing config screen on the main router.

Maurice, could this overlap with the blackhole route we added for the delegated prefix?


Cheers,
Franco

[Maurice]

@franco That would be weird, since the automatic blackhole route and the static routes have different prefix lengths. Adding a /48 blackhole route should not remove existing routes for /60 subnets. But this should be easy to test by creating static routes for prefixes which aren't subnets of the delegated prefix.

@matt335672, what's your WAN configuration, static or DHCPv6?

And I reconsidered what I said about having observed this before. What I have indeed observed is some static routes sometimes not getting added to the routing table after a reboot. But I think these were static IPv4 routes on ptp interfaces, so probably a different issue.

Cheers
Maurice

[matt335672]

I've got a blackhole route manually added anyway - it's been there a few years.

WAN on the main router is DHCPv6 (PPPoE)

A bit more info - I just spent a couple of hours trying to reproduce this on a VM with two interfaces, and failed. I then started looking at the clean config on the VM, and my fairly crufty config on the primary router, saving configs as I went along.

I had a stale reservation in Kea for a GUA on the secondary router. Although the secondary router is not now using DHCPv6 for configuration, when I removed the reservation the routing problem went away.

Difference in the configs is:

Code Select Expand

@@ -1561,9 +1561,9 @@
    <column_count>2</column_count>
  </widgets>
  <revision>
-    <username>opnadmin@{prefix48}:49:323e:ff94:6713:d8b5</username>
-    <description>/api/routes/routes/toggleroute/232ecc2c-722f-4390-84ec-285d8bb15f5d made changes</description>
-    <time>1768837329.21</time>
+    <username>opnadmin@{prefix48}:49:27b7:e3d6:48bb:a479</username>
+    <description>/api/kea/dhcpv6/set made changes</description>
+    <time>1769099743.71</time>
  </revision>
  <OPNsense>
    <Firewall>
@@ -2774,7 +2774,7 @@
        </reservations>
        <ha_peers/>
      </dhcp4>
-      <dhcp6 version="1.0.0" persisted_at="1768836386.51" description="Kea DHCPv6 configuration">
+      <dhcp6 version="1.0.0" persisted_at="1769099743.71" description="Kea DHCPv6 configuration">
        <general>
          <enabled>1</enabled>
          <manual_config>0</manual_config>
@@ -2876,14 +2876,6 @@
          </subnet6>
        </subnets>
        <reservations>
-          <reservation uuid="2d620bbf-fe2f-4e43-b186-1a5e3f6f0048">
-            <subnet>7c128662-0440-464c-a14d-844667d1cfa4</subnet>
-            <ip_address>{prefix48}:4f::1fff</ip_address>
-            <duid>00:01:00:01:26:dd:37:1c:52:54:00:68:30:cf</duid>
-            <hostname>router2</hostname>
-            <domain_search/>
-            <description/>
-          </reservation>
          <reservation uuid="39055ba4-31b6-4409-a38b-df0a91fceecb">
            <subnet>d0d43b13-dc16-4c61-9eaf-31417bcbfbe7</subnet>
            <ip_address>{prefix48}:49::a</ip_address>

However, I'm unable now to get the old config to fail. If I reload it, everything seems to be working OK. Either I've made a mistake in my notes, or there's something else going on.

I've tried adding a reservation to my test VM, but that doesn't fail either.

This probably isn't a very useful data point, I'm afraid, as I'm unable to get back to a non-working configuration now.

[franco]

If there's a bug it will come back to be looked at. We can wait.  :)


Cheers,
Franco