[SOLVED] Adding IPv6 static route for rc.newwanipv6 fail

[bringha]

Hello,

after having fixed so fast (thanks again Franco !!) the matter around apinger (see https://forum.opnsense.org/index.php?topic=6028.0), there is another new error message in my system.log, which seems to have on a first look a similar root cause:

Code: [Select]

OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Removing static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX
OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Adding static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX
OPNsense opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -host -'inet6' 'fe80::3631:c4ff:XXXX:XXXX%igb1' 'fe80::3631:c4ff:XXXX:XXXX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add host fe80::3631:c4ff:XXXX:XXXX%igb1: gateway fe80::3631:c4ff:XXXX:XXXX fib 0: Network is unreachable'

Could it be that the link local address of the default ipv6 gateway is not assigned to my WAN interface too for the adding the static route which would require some adoption of the rc.newwanipv6?

Looking forward to your reply

Br br

[bringha]

After some more analysis work, it seems to be related to https://forum.opnsense.org/index.php?topic=6028.0)

It affects the function system_host_route in /usr/local/etc/inc/system.inc which is called for the routes to DNS servers and monitors.

Code: [Select]

function system_host_route($host, $gateway, $delete = true, $add = true)
{
    if (is_ipaddrv4($gateway)) {
        $family = 'inet';
    } elseif (is_ipaddrv6($gateway)) {
        $family = 'inet6';
    } else {
        return;
    }

    if ($delete) {
        mwexecf('/sbin/route delete -host -%s %s', array($family, $host), true);
    }

    if ($add) {
-->        /* Added by bringha for ipv6   */
-->        if ($family == "inet6" && (is_linklocal($gateway))) {
-->            $interface = get_real_interface("wan");
-->            $gateway .= "%{$interface}";
-->        }
        mwexecf('/sbin/route add -host -%s %s %s', array($family, $host, $gateway));
    }
}


I added the lines ---> for my workaround to get access to the ipv6 dns servers  I have configured (otherwise the route is not added) and the monitor (for the latter the error message is related to).

This might not be the best code. I would think that it is better to add the parameter $interface to the function when called. Then it is consistent to e.g. all the other routing functions again (e.g. system_default_route($gateway, $interface = null, $delete = true, $add = true)). The callers need to be adapted then in system.inc and in gwlb.inc accordingly.

Anyway, with this patch, the log file looks much better

Code: [Select]

Sep 24 20:23:37 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'igb1'
Sep 24 20:23:37 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::217:3fff:XXXX:XXXX) (interface: WAN[wan]) (real interface: igb1).
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv4 default route to 192.168.X.X
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::3631:c4ff:XXXX:XXXX
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Removing static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Adding static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX

Br br

[franco]

... but we shouldn't open new topics when the problems are clearly related

The fix is not entirely correct here, "wan" is not always the correct interfaces. We need to pass the interface from the gateway code as well just like with the apinger.conf.

[franco]

How about this then?

https://github.com/opnsense/core/commit/c6c9d1029

[bringha]

Hi Franco,

apologies I was not sure whether it is really related as the same error message is also appearing for the route to the ipv6 DNS service

Code: [Select]

Sep 24 12:17:49 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'igb1'
Sep 24 12:17:49 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::217:3fff:XXXX:XXXX) (interface: WAN[wan]) (real interface: igb1).
Sep 24 12:17:53 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -host -'inet6' '2001:470:20::2' 'fe80::3631:c4ff:XXXX:XXXX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add host 2001:470:20::2: gateway fe80::3631:c4ff:XXXX:XXXX fib 0: Network is unreachable'
Sep 24 12:17:53 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv4 default route to 192.168.X.X
Sep 24 11:42:57 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -host -'inet6' '2001:470:20::2' 'fe80::3631:c4ff:XXXX:XXXX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add host 2001:470:20::2: gateway fe80::3631:c4ff:XXXX:XXXX fib 0: Network is unreachable'sr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::3631:c4ff:XXXX:XXXX
The problem is obviously somewhat broader: function system_host_route() is also called to make the static routes for the DNS service in system.inc, 224ff; it this was the reason

• That I opened up a new topic 
• that I extended the function directly

When I understand your patch correctly, then you address the problem with the monitor, the DNS problem will not benefit from it. Still therefore the suggestion to patch it like handing over a $interface parameter to system_host_route(); then you can also use the 'right one' and not as dirty as I did ....

Just my 10 cents ....

Br br

[franco]

I'm confused, the output you now post is a different one than before. "adding monitor" was gateway code for sure, now it isn't.

Do you have a static route set up as well?

system_host_route should be invoked with a functional address, so we fix the callers, not the function and the callers


Cheers,
Franco

[bringha]

Thanks Franco,

This is exactely what I tried to say:

Monitor and DNS both are calling /sbin/route out of system_host_route() and if your ipv6 gateway is link local and does not contain a %$interface after the LL address, then the command ends in an error message; but indeed monitor and DNS both serving different purposes

Beyond of that I don't use explicit static routes. My understanding is that configured DNS servers are added to the routing table statically when the system is started !?

I also suggested - instead of my really somewhat odd hack  -  to code it on the callers side - appreciated; Never the less my dirty hack show that adding the %$interface thing solves the problem ....

Apologies if I have expressed myself too complicated ....

Br br

[franco]

Apologies from my side as well, I am only trying not to miss anything.

Ironically, static DNS routes also track their way back to gwlb.inc...

https://github.com/opnsense/core/commit/72edd79

That should be it.


Cheers,
Franco

[bringha]

Thanks Franco,

have now installed both patches, will report whether they solve the issue ....

Br br

[bringha]

Hi Franco,

Hm .... DNS still does not work

could it be that line 905 which is

Code: [Select]

if (is_linklocal($gw['gateway']) && !strstr($gw['gateway'], '%') === false) {


should look like either

Code: [Select]

if (is_linklocal($gw['gateway']) && !strstr($gw['gateway'], '%')) {

OR

Code: [Select]

if (is_linklocal($gw['gateway']) && strpos($gw['gateway'], '%') === false) {
?

BR br

[franco]

Hi br,

Yes, good catch. A copy+paste error while fidding with branches:

https://github.com/opnsense/core/commit/e3dcbf8

Fourth time is the charm?!


Cheers,
Franco

[bringha]

Hi Franco,

As easy as that - so far no further error!

And even more - so far the first Sense ever which does now supporting fill ipv6 across four networks at least for my current (pretty straight) config at a dual stack ISP

Br br

[franco]

Hi br,

Sounds good. Had the chance today to deploy this on a IPv6 install today. Only change that I see is that the gateways widget now shows the scope of local addresses, but I would consider this a feature.

I will do more tests with static DNS later, but it's good enough for commit to master now.


Thanks,
Franco

[bringha]

... one question - the code is not contained in 17.7.4?

Br br

[franco]

Yes, it needs a manual apply as it was on a separate branch during the build process of 17.7.4, but it's now a single commit on the development branch:

https://github.com/opnsense/core/commit/f77ab1b3

I'm unsure about timing as 17.7.5 brings new images, so we would want to wait with this till 17.7.6.


Cheers,
Franco