[solved] LAN and OPT1 interface are getting the same link local address

[guest19228]

Hello, I'm new at OPNsense. I was using m0n0wall as long as it was actively developed and switched then to openwrt. Now I want to give OPNsense as a successor of m0n0wall a try. I'm using OPNsense 18.7.5-amd64 FreeBSD 11.1-RELEASE-p14 LibreSSL 2.7.4. After a lot of struggling I finally managed to get an interface configuration that looks working. (Compared to openwrt the IPv6 setup in OPNsense is a real pain in the ass) However for some reasons while the WAN interface is getting an correct one the LAN and OPT1(DMZ) interfaces are getting the same link local address. (see below)
WAN interface

Code: [Select]

vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 52:54:00:21:46:95
        hwaddr 52:54:00:21:46:95
        inet6 fe80::5054:ff:fe21:4695%vtnet0 prefixlen 64 scopeid 0x1
        inet6 2002:xxxx:4695 prefixlen 64 autoconf
        inet6 2002:xxxx:8f86 prefixlen 64 autoconf temporary
        inet xxx.xxx.xxx.xxx netmask 0xffffff00 broadcast xxx.xxx.xxx.xxx
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: activeLAN interface

Code: [Select]

vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 52:54:00:42:42:96
        hwaddr 52:54:00:42:42:96
        inet xxx.xxx.xxx.xxx netmask 0xffffff00 broadcast xxx.xxx.xxx.xxx
        inet6 2002:xxxx:4296 prefixlen 64
        inet6 fe80::1:1%vtnet1 prefixlen 64 scopeid 0x2
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: activeand DMZ interface

Code: [Select]

vtnet2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 52:54:00:8d:d1:37
        hwaddr 52:54:00:8d:d1:37
        inet xxx.xxx.xxx.xxx netmask 0xffffff00 broadcast xxx.xxx.xxx.xxx
        inet6 2002:xxxx:d137 prefixlen 64
        inet6 fe80::1:1%vtnet2 prefixlen 64 duplicated scopeid 0x3
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: activeAs you can see the WAN interface is getting an fe80:: address derived from the MAC address as it should be, while the LAN and DMZ interfaces are getting both fe80::1:1 as link local address.
What is going wrong here?

[bringha]

Nothing goes wrong - all fine!

fe80::1:1 is the link local ipv6 default gateway address. With the zoneid extension %<interface> its getting properly differentiatable. The WAN has indeed no default gateway on its own interface ...

Background see eg here

https://www.edge-cloud.net/2013/08/07/ipv6-link-local-addresses-as-default-gateway/amp/

Br br

[franco]

You're tracking WAN IPv6 from LAN and OPT1 at the same time. This is expected.


Cheers,
Franco

[guest19228]

Do I understand right, the reason for manually assigning both nics the same address fe80::1:1 is the tracking of the wan interface? Is there no other way possible? I'm currently still using openwrt and there the tracking or whatever they do to assign public IPv6 addresses to the lan and dmz interfaces works with the link local address assigend by the kernel.
So in openwrt it looks like this:
WAN interface:

Code: [Select]

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc cake state UP group default qlen 1000
    link/ether 52:54:00:28:c2:19 brd ff:ff:ff:ff:ff:ff
    inet 192.168xxx.xxx/24 brd 192.168.xxx.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2002:XXXX:c219/64 scope global dynamic noprefixroute
       valid_lft 6951sec preferred_lft 3351sec
    inet6 fe80::5054:ff:fe28:c219/64 scope link
       valid_lft forever preferred_lft foreverLAN interface:

Code: [Select]

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:00:b0:42 brd ff:ff:ff:ff:ff:ff
    inet 192.xxx.xxx.xxx/24 brd 192.xxx.xxx.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 2002:XXXX:1d3a/62 scope global dynamic noprefixroute
       valid_lft 7132sec preferred_lft 3532sec
    inet6 fd45:887:44b9:4:ab5d:91d8:dc:106b/62 scope global noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe00:b042/64 scope link
       valid_lft forever preferred_lft foreverthe DMZ interface:

Code: [Select]

4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:76:d3:d2 brd ff:ff:ff:ff:ff:ff
    inet 192.xxx.xxx.xxx/24 brd 192.xxx.xxx.255 scope global eth2
       valid_lft forever preferred_lft forever
    inet6 2002:XXXX:e75e/62 scope global dynamic noprefixroute
       valid_lft 7001sec preferred_lft 3401sec
    inet6 fd45:887:44b9::215:bc8:af91:fe16/62 scope global noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe76:d3d2/64 scope link
       valid_lft forever preferred_lft foreverThe config file for nic:

Code: [Select]

config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'

config globals 'globals'
option ula_prefix 'fd45:0887:44b9::/48'

config interface 'wan'
option ifname 'eth0'
option proto 'dhcp'
option peerdns '0'
option dns '84.200.69.80 84.200.70.40 194.150.168.168'

config interface 'wan6'
option ifname 'eth0'
option proto 'dhcpv6'
option reqaddress 'try'
option reqprefix '60'
option peerdns '0'
option dns '2001:1608:10:25::1c04:b12f 2001:1608:10:25::9249:d69b'

config interface 'lan'
option ifname 'eth1'
option proto 'static'
option ipaddr '192.xxx.xxx.xxx'
option netmask '255.255.255.0'
option ip6assign '62'
option ip6ifaceid 'random'

config interface 'dmz'
option proto 'static'
option ifname 'eth2'
option ipaddr '192.xxx.xxx.xxx'
option netmask '255.255.255.0'
option ip6ifaceid 'random'
option ip6assign '62' With this config it works like a charm for several years now.

The problem I'm seeing is the fe80::1:1 address assigned to the dmz interface is marked as duplicated:

Code: [Select]

inet6 fe80::1:1%vtnet2 prefixlen 64 duplicated scopeid 0x3That should not happen.
When using the link local address assigned by the kernel this would never happen.
In https://en.wikipedia.org/wiki/IPv6_address I found this:

For addresses with other than global scope (as described below), and in particular for link-local addresses, the choice of the network interface for sending a packet may depend on which zone the address belongs to: the same address may be valid in different zones, and be in use by a different host in each of those zones. Even if a single address is not in use in different zones, the address prefixes for addresses in those zones may still be identical, which makes the operating system unable to select an outgoing interface based on the information in the routing table (which is prefix-based).

In order to resolve the ambiguity in textual addresses, a zone index must be appended to the address, the two separated by a percent sign (%).[10] The syntax of zone indices is an implementation-dependent string, although numeric zone indices must be universally supported as well. The following link-local address:

    fe80::1ff:fe23:4567:890a

could become for instance:

    fe80::1ff:fe23:4567:890a%eth2

or:

    fe80::1ff:fe23:4567:890a%3

The former (using an interface name) is customary on most Unix-like operating systems (e.g., BSD, Linux, OS X). The latter (using an interface number) is the standard syntax on Microsoft Windows, but as support for this syntax is mandatory, it is also available on other operating systems.

BSD-based operating systems (including OS X) also support an alternative, non-standard syntax, where a numeric zone index is encoded in the second 16-bit word of the address. E.g.:

    fe80:3::1ff:fe23:4567:890a

In all operating systems mentioned above, the zone index for link-local addresses actually refers to an interface, not to a zone. As multiple interfaces may belong to the same zone (e.g. when connected to the same switch), in practice two addresses with different zone-ids may actually be equivalent, and refer to the same host on the same link.

Duplicate address detection

The assignment of a unicast IPv6 address to an interface involves an internal test for the uniqueness of that address using Neighbor Solicitation and Neighbor Advertisement (ICMPv6 type 135 and 136) messages. While in the process of establishing uniqueness an address has a tentative state.

The node joins the solicited-node multicast address for the tentative address (if not already done so) and sends neighbor solicitations, with the tentative address as target address and the unspecified address (::/128) as source address. The node also joins the all-hosts multicast address ff02::1, so it will be able to receive Neighbor Advertisements.

If a node receives a neighbor solicitation with its own tentative address as the target address, then that address is not unique. The same is true if the node receives a neighbor advertisement with the tentative address as the source of the advertisement. Only after having successfully established that an address is unique may it be assigned and used by an interface. 

So because the fe80:: address on the dmz interfaces is marked as duplicated it may not work but a working link local address is required for NDP DHCP6 and some other protocols. This could easily be prevented when using the link local address assigned by the kernel.

[bringha]

Hi emwe,

here for comparison my config with 4 nics also using tracking of the WAN and having 4 different scopeid's. WAN is igb1. As you can see, all my internal interfaces have an fe80::1:1 with translated scopeid/zoneindex according to the standard.

However, my interfaces do not show the 'duplicated' attribute for the link local address. Interesting to find out from where this is coming from. Never seen that before for an LL Address with scopeID. As it is also described in your wikipedia article, differentiation of equal LL addresses are done with the zoneID. Could it be that you have the same scopeID on different network interfaces or that dmz and lan are bridged (eg via switch)?

Code: [Select]

igb0: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
ether 00:XX:1a
hwaddr ac:XX:98
inet 192.168.XX.XX netmask 0xffffff00 broadcast 192.168.XX.255
inet6 2003:XX:a21a prefixlen 64
inet6 fe80::1:1%igb0 prefixlen 64 scopeid 0x1
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
igb1: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
ether 00:XX:1d
hwaddr ac:XX:99
inet6 fe80::XX:a21d%igb1 prefixlen 64 scopeid 0x2
inet6 2003:XX:febe:a21d prefixlen 64 autoconf
inet 192.168.XX.XX netmask 0xffffff00 broadcast 192.168.XX.255
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
ether 00:XX:c7
hwaddr ac:XX:9a
inet 192.168.XX.1 netmask 0xffffff00 broadcast 192.168.XX.255
inet6 2003:XX:4bc7 prefixlen 64
inet6 fe80::1:1%igb2 prefixlen 64 scopeid 0x3
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
igb3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
ether 00:XX:c6
hwaddr ac:XX:9b
inet 10.XX.XX.1 netmask 0xffffff00 broadcast 10.XX.XX.255
inet6 2003:XXXX:4bc6 prefixlen 64
inet6 fe80::1:1%igb3 prefixlen 64 scopeid 0x4
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active

Br br

[guest19228]

Hello,
I did a lot of checks to find out why fe80::1:1 is marked as duplicated on vtnet2. The only device where this address is used is indeed the opnsense installation. Further investigations brought up, that the switch is the reason. Several years ago I created an isolated port group for DMZ on it but probably forgot to save the running configuration into the startup configuration. So after a powerloss the switch no longer had that isolated port group. So the LAN and the DMZ interface could "see" each other via the switch. Oddly enough it never caused any problem until now. I now recreated that isolated port group on the switch and the duplicate mark is gone. THis time I made sure it is saved into the startup configuration.

On the other hand I'm not really lucky wiht the behaviour of opnsense to overwrite the automatically assigned link local address with fe80::1:1 and searched through the configuration scripts to find a reason why it is done.

I found it in /usr/local/etc/inc7interfaces.inc lines 2573 to 2581.

Code: [Select]

    /* always configure a link-local of fe80::1:1 on the track6 interfaces */
    /* $realif = get_real_interface($interface);
    * $linklocal = find_interface_ipv6_ll($realif);
    * if (!empty($linklocal)) {
    *    mwexec("/sbin/ifconfig {$realif} inet6 {$linklocal} delete");
    }*/
    /* XXX: This might break for good on a carp installation using link-local as network ips */
    /* XXX: Probably should remove? */
    /*mwexec("/sbin/ifconfig {$realif} inet6 fe80::1:1%{$realif}");*/This is the only occurence of fe80::1:1 in all configuration scripts and so I decided to comment it out. The result is as expected. All interfaces do now have an fe80:: address with a suffix generated from the interface mac address. And the WAN tracking also still works.

Code: [Select]

vtnet2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 52:54:00:8d:d1:37
        hwaddr 52:54:00:8d:d1:37
        inet 192.168.5.1 netmask 0xffffff00 broadcast 192.168.5.255
        inet6 fe80::5054:ff:fe8d:d137%vtnet2 prefixlen 64 scopeid 0x3
        inet6 2002:bc67:2360:d3:5054:ff:fe8d:d137 prefixlen 64
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: activeThe lines
/* XXX: This might break for good on a carp installation using link-local as network ips */
/* XXX: Probably should remove? */
show that someone was already aware that manully assigning fe80::1:1 to the interfaces is probably not a good idea. So why no remove this snippet or modify it in a way that it will only assign a link local address when none is already assigned?

[bringha]

Hi emwe,

If its for your configuration a good decision to give your sense a different LL address - fine...  but I doubt that this is a good solution in general:

There is a widely accepted convention in ipv6 network design that LL address fe80::1:1%<zoneid> is used as default ipv6 gateway address. See eg here again: https://www.edge-cloud.net/2013/08/07/ipv6-link-local-addresses-as-default-gateway/amp/, section 'ipv6 use case'.

Having a textual representation of fe80::1:1%<zoneid> for this address is all what you need. As with that all autoconf ipv6 protocols work like a charm I am wondering rather vice versa, what use case behind really REQUIRES to have a different LL address on the interfaces of our router node in your network. The cited comment from the source around CARP is something which is according my understanding not really a practical problem yet ...

I personally rather tend to explicitly APPRECIATE that all my LANs have a standard ipv6 gateway address so that setting the default gateway in pure autoconf setups does not require extra 'research' to find the specific LL address of the sense interface .... One of my sense has btw 8 NICs and there might be even bigger scenarios ...

Historically, a similar convention is also in ipv4, where in private address space eg 192.168.X.1 and 192.168.X.254 are the recommended router/gateway addresses

Br br

[guest19228]

Hello bringha,
imagine the following scenario. You have a network with 2 opnsense firewalls. Each is acting as a router for a different address range. If now both firewalls have the same fe80::1:1 address on the same physical network, you have the classical duplicated IP address problem.

[bringha]

Hi emwe

By definition there can only be one default gateway per routing table. Same is true for IPv4. if you want to route traffic to several gateways use source based routing and create additional routing tables with default routes specific per table. A roule is required which router to use with what source address.

As also stated in the link above, next chapter network design:

As each IPv6-enabled router interface will already have a link-local address generated based on the modified EUI-64 scheme, the address fe80::1 will either replace or augment this automatically generated address. At the same time an existing global unicast address on the interface will not be affected.

Have not tested yet whether OPNsense behaves like that; anyhow, this would be the more compliant solution ....

Br br

[guest19228]

Hello bringha,
ofcourse you are right, there can be only one default gateway and if you want to use another for a specific IP range ouy hafe to define an additional gateway for it. But this is easy done via dhcp option 121(i.e 121,224.0.0.0/4,192.168.4.1,0.0.0.0/0,192.168.4.1) I had to use this construct because I had more than one network card in my PC and vlc was always choosing the wrong one for IP TV Also the hardware mediaplayer I got from my ISP did not work when behind the firewall (with active igmp proxy) but did when attached directly to the dsl router. Adding this option to the dhcp server solved the problem. But should be discussed in another topic if interested.
But let's end this disussion. My main problem is that deleting the kernel assigned LL address and replacing it with fe80::1:1 is currently hard coded. This leaves you no choice. Take this as a suggestion for the developers. It would be much better to change the code in a way that you can choose if you want to overwrite the LL address or simply add an addtional one and also let you choose the LL address to use. Ofcourse this requires a lot more more code and makes it more complex but you have more flexibility and will help to avoid duplicated IP address confilcts.

Many thanks for your nice explanations and hints.

[franco]

I am unclear as to what the issue is. If I understand correctly:

LAN and DMZ are both switched into the same network creating a link-local address overlap for clients?

In this case the solution is to physically separate LAN and DMZ switching, no?


Cheers,
Franco

[guest19228]

Hello franco,
I'm afraid you missunderstood. It was never my intention to have LAN and DMZ in the same network. That it happened was my fault due to a lost configuration on the switch. The problem concerning me now is not longer the original one, so I started the new topic https://forum.opnsense.org/index.php?topic=10034.0 therefore.