OPNsense Forum

Archive => 17.7 Legacy Series => Topic started by: bringha on September 24, 2017, 04:41:42 pm

Title: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 24, 2017, 04:41:42 pm
Hello,

after having fixed so fast (thanks again Franco !!) the matter around apinger (see https://forum.opnsense.org/index.php?topic=6028.0 (https://forum.opnsense.org/index.php?topic=6028.0)), there is another new error message in my system.log, which seems to have on a first look a similar root cause:

Code: [Select]
OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Removing static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX
OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Adding static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX
OPNsense opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -host -'inet6' 'fe80::3631:c4ff:XXXX:XXXX%igb1' 'fe80::3631:c4ff:XXXX:XXXX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add host fe80::3631:c4ff:XXXX:XXXX%igb1: gateway fe80::3631:c4ff:XXXX:XXXX fib 0: Network is unreachable'

Could it be that the link local address of the default ipv6 gateway is not assigned to my WAN interface too for the adding the static route which would require some adoption of the rc.newwanipv6?

Looking forward to your reply

Br br
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 24, 2017, 07:06:43 pm
After some more analysis work, it seems to be related to https://forum.opnsense.org/index.php?topic=6028.0) (https://forum.opnsense.org/index.php?topic=6028.0))

It affects the function system_host_route in /usr/local/etc/inc/system.inc which is called for the routes to DNS servers and monitors.
Code: [Select]
function system_host_route($host, $gateway, $delete = true, $add = true)
{
    if (is_ipaddrv4($gateway)) {
        $family = 'inet';
    } elseif (is_ipaddrv6($gateway)) {
        $family = 'inet6';
    } else {
        return;
    }

    if ($delete) {
        mwexecf('/sbin/route delete -host -%s %s', array($family, $host), true);
    }

    if ($add) {
-->        /* Added by bringha for ipv6   */
-->        if ($family == "inet6" && (is_linklocal($gateway))) {
-->            $interface = get_real_interface("wan");
-->            $gateway .= "%{$interface}";
-->        }
        mwexecf('/sbin/route add -host -%s %s %s', array($family, $host, $gateway));
    }
}


I added the lines ---> for my workaround to get access to the ipv6 dns servers  I have configured (otherwise the route is not added) and the monitor (for the latter the error message is related to).

This might not be the best code. I would think that it is better to add the parameter $interface to the function when called. Then it is consistent to e.g. all the other routing functions again (e.g. system_default_route($gateway, $interface = null, $delete = true, $add = true)). The callers need to be adapted then in system.inc and in gwlb.inc accordingly.

Anyway, with this patch, the log file looks much better
Code: [Select]
Sep 24 20:23:37 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'igb1'
Sep 24 20:23:37 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::217:3fff:XXXX:XXXX) (interface: WAN[wan]) (real interface: igb1).
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv4 default route to 192.168.X.X
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::3631:c4ff:XXXX:XXXX
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Removing static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX
Sep 24 20:23:42 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: Adding static route for monitor fe80::3631:c4ff:XXXX:XXXX%igb1 via fe80::3631:c4ff:XXXX:XXXX

Br br
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 25, 2017, 08:30:53 am
... but we shouldn't open new topics when the problems are clearly related :D

The fix is not entirely correct here, "wan" is not always the correct interfaces. We need to pass the interface from the gateway code as well just like with the apinger.conf.
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 25, 2017, 10:13:03 am
How about this then?

https://github.com/opnsense/core/commit/c6c9d1029
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 25, 2017, 12:26:53 pm
Hi Franco,

apologies I was not sure whether it is really related as the same error message is also appearing for the route to the ipv6 DNS service
Code: [Select]
Sep 24 12:17:49 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'igb1'
Sep 24 12:17:49 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::217:3fff:XXXX:XXXX) (interface: WAN[wan]) (real interface: igb1).
Sep 24 12:17:53 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -host -'inet6' '2001:470:20::2' 'fe80::3631:c4ff:XXXX:XXXX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add host 2001:470:20::2: gateway fe80::3631:c4ff:XXXX:XXXX fib 0: Network is unreachable'
Sep 24 12:17:53 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv4 default route to 192.168.X.X
Sep 24 11:42:57 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -host -'inet6' '2001:470:20::2' 'fe80::3631:c4ff:XXXX:XXXX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add host 2001:470:20::2: gateway fe80::3631:c4ff:XXXX:XXXX fib 0: Network is unreachable'sr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::3631:c4ff:XXXX:XXXX
The problem is obviously somewhat broader: function system_host_route() is also called to make the static routes for the DNS service in system.inc, 224ff; it this was the reason

When I understand your patch correctly, then you address the problem with the monitor, the DNS problem will not benefit from it. Still therefore the suggestion to patch it like handing over a $interface parameter to system_host_route(); then you can also use the 'right one' and not as dirty as I did ....

Just my 10 cents ....

Br br
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 25, 2017, 01:24:50 pm
I'm confused, the output you now post is a different one than before. "adding monitor" was gateway code for sure, now it isn't. :D

Do you have a static route set up as well?

system_host_route should be invoked with a functional address, so we fix the callers, not the function and the callers :)


Cheers,
Franco
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 25, 2017, 02:04:05 pm
Thanks Franco,

This is exactely what I tried to say:

Monitor and DNS both are calling /sbin/route out of system_host_route() and if your ipv6 gateway is link local and does not contain a %$interface after the LL address, then the command ends in an error message; but indeed monitor and DNS both serving different purposes :)

Beyond of that I don't use explicit static routes. My understanding is that configured DNS servers are added to the routing table statically when the system is started !?

I also suggested - instead of my really somewhat odd hack  ::) -  to code it on the callers side - appreciated; Never the less my dirty hack show that adding the %$interface thing solves the problem ....

Apologies if I have expressed myself too complicated ....

Br br
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 25, 2017, 02:55:39 pm
Apologies from my side as well, I am only trying not to miss anything.

Ironically, static DNS routes also track their way back to gwlb.inc...

https://github.com/opnsense/core/commit/72edd79

That should be it. :)


Cheers,
Franco
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 25, 2017, 03:13:51 pm
Thanks Franco,

have now installed both patches, will report whether they solve the issue ....

Br br
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 25, 2017, 08:13:32 pm
Hi Franco,

Hm .... DNS still does not work

could it be that line 905 which is

Code: [Select]
if (is_linklocal($gw['gateway']) && !strstr($gw['gateway'], '%') === false) {


should look like either

Code: [Select]
if (is_linklocal($gw['gateway']) && !strstr($gw['gateway'], '%')) {

OR

Code: [Select]
if (is_linklocal($gw['gateway']) && strpos($gw['gateway'], '%') === false) {
?

BR br
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 25, 2017, 10:22:38 pm
Hi br,

Yes, good catch. A copy+paste error while fidding with branches:

https://github.com/opnsense/core/commit/e3dcbf8

Fourth time is the charm?! :)


Cheers,
Franco
Title: Re: Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 26, 2017, 08:47:59 am
Hi Franco,

As easy as that - so far no further error!

And even more - so far the first Sense ever which does now supporting fill ipv6 across four networks at least for my current (pretty straight) config at a dual stack ISP

Br br
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 26, 2017, 11:21:51 am
Hi br,

Sounds good. Had the chance today to deploy this on a IPv6 install today. Only change that I see is that the gateways widget now shows the scope of local addresses, but I would consider this a feature.

I will do more tests with static DNS later, but it's good enough for commit to master now.


Thanks,
Franco
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 27, 2017, 10:52:12 pm
... one question - the code is not contained in 17.7.4?

Br br
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 28, 2017, 08:20:57 am
Yes, it needs a manual apply as it was on a separate branch during the build process of 17.7.4, but it's now a single commit on the development branch:

https://github.com/opnsense/core/commit/f77ab1b3

I'm unsure about timing as 17.7.5 brings new images, so we would want to wait with this till 17.7.6.


Cheers,
Franco
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 28, 2017, 10:35:24 am
Thanks Franco,

as from my perspective this change is a very basic enabler to all users who are working with ipv6 in conjunction with dual stack providers and a standard router/modem like fritzbox et al towards the ISPs the sooner the better ....
if not that some new findings leave some doubt that this code has the required stability ...

BR C
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on September 28, 2017, 03:26:57 pm
I agree. Learning from the past we reluctantly push changes to the master branch that we have not tested in a real environment. Tuesday was the first day that I could test, so it wasn't shipped with 17.7.4, neither in release nor development version.

So far it seems to be a solid change and it will be considered for release with 17.7.5 unless something unexpected happens.


Cheers,
Franco
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on September 28, 2017, 04:42:50 pm
 :)

I let it run in my productive environment up to then and let you know if something particular happens too

BR Br
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: franco on October 02, 2017, 06:59:54 pm
Hi C,

Added to stable/17.7 so it will land in 17.7.5.


Cheers,
Franco
Title: Re: [SOLVED] Adding IPv6 static route for rc.newwanipv6 fail
Post by: bringha on October 02, 2017, 09:46:18 pm
Super, thats great!

No further error also on my productive environment, stable gw monitoring and static routes to DNS;

Br br