After Upgrade to 24.7.4 Zerotier not working

[Mann-IT]

I do this on 24.7.4

and it help!!!!

root@OPNsense:~ # opnsense-patch 1dba25fed8
Fetched 1dba25fed8 via https://github.com/opnsense/core
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From 1dba25fed8686f865a9425bd08c04a01075c94e1 Mon Sep 17 00:00:00 2001
|From: Franco Fichtner <franco@opnsense.org>
|Date: Fri, 7 Jun 2024 22:22:02 +0200
|Subject: [PATCH] interfaces: force regeneration of link-local on spoofed MAC;
| closes #4430
|
|(cherry picked from commit a2ac1999f37ee98da22b6edd42c430c8dbb6534b)
|(cherry picked from commit 7669567944ec26ffea088a636482e04b0e9912d6)
|---
| src/etc/inc/interfaces.inc | 35 +++++++++++++++++++++++++++++++----
| 1 file changed, 31 insertions(+), 4 deletions(-)
|
|diff --git a/src/etc/inc/interfaces.inc b/src/etc/inc/interfaces.inc
|index 1b672e7f50..152155dafa 100644
|--- a/src/etc/inc/interfaces.inc
|+++ b/src/etc/inc/interfaces.inc
--------------------------
Patching file etc/inc/interfaces.inc using Plan A...
Reversed (or previously applied) patch detected!  Assuming -R.Hunk #1 succeeded at 2316 (offset 32 lines).
done
All patches have been applied successfully.  Have a nice day.

[dalton_lopes]

opnsense-patch 1dba25fed8

it solved to me!

Thanks, Franco.

[edgehoax]

Hi,

works a treat !

Thanks Franco !

[you]

Confirmed. Patch works here as well. Thanks

[newsense]

Confirmed working with the patch.

[pbk]

# opnsense-patch 1dba25fed8

Applied to 24.7.4 and ZT is working again after a reboot.

[franco]

The patch is just for triage. ZeroTier has an issue with auto-link-local flag and I have no way of testing this so someone with the setup please take a closer look at ifconfig in the working and non-working case.

My assumption is still that this is true for assigned ZeroTier interfaces, but maybe I missed someone confirming that. And is this an IPv4 or IPv6 tunnel?


Cheers,
Franco

[dalton_lopes]

i'm using only IPv4 tunnel

[pbk]

I have no way of testing this

I can set up a public ZT network for you to play with, just drop me a line

so someone with the setup please take a closer look at ifconfig in the working and non-working case.

Just the ZT part:

Non working:

Code Select Expand


REDACTED: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 5000 mtu 2800
description: ZeroTier (opt2)
options=80000<LINKSTATE>
ether 58:9c:fc:10:92:2f
inet 172.27.8.25 netmask 0xffff0000 broadcast 172.27.255.255
inet6 fe80::5a9c:ffff:ffff:ffff%REDACTED prefixlen 64 scopeid 0x7
groups: tap
media: Ethernet 1000baseT <full-duplex>
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Opened by PID 66352



Working after applying the patch:

Code Select Expand


REDACTED: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 5000 mtu 2800
description: ZeroTier (opt2)
options=80000<LINKSTATE>
ether 7a:fd:ba:es:1f:1c
hwaddr 58:9c:fc:10:92:2f
inet 172.27.8.25 netmask 0xffff0000 broadcast 172.27.255.255
inet6 fe80::5a9c:ffff:ffff:ffff%REDACTED prefixlen 64 scopeid 0x7
groups: tap
media: Ethernet 1000baseT <full-duplex>
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Opened by PID 61635


My assumption is still that this is true for assigned ZeroTier interfaces, but maybe I missed someone confirming that.

The ZT networks are assigned to an interface in my case, yes.

And is this an IPv4 or IPv6 tunnel?

IPv4 in the tunnel

[franco]

Thanks, I see the issue is somewhat similar to LAGG interfaces: ZeroTier modifies the Ethernet address of the device on its own. That certainly isn't great. I'll propose a patch next week.


Cheers,
Franco

[pbk]

ZeroTier modifies the Ethernet address of the device on its own.

It has to. Each device in a ZT network has its own MAC which is calculated from the member id of that device. This address does not change as long as the member id doesn't change which it only does if someone manually resets the member id and therefore makes it a new device to ZT.

ZT needs its own MAC because it works as a SDWAN switch and needs arp to function.

That answers the question why nothing arrived at the firewall. It was just impossible to send Ethernet frames to the ZT network member MAC from OPNsense.

[newsense]

FYI, 24.7.4_1 only fixes a PPP regression, not the ZT issue.


If in need of the PPP fix make sure to reapply the patch Franco posted earlier.

Otherwise here's no need to do anything until Franco has the ZT patch out.

[franco]

ZT needs its own MAC because it works as a SDWAN switch and needs arp to function.

While that seems clear it's also broken by design from the start because you could override the MAC address from the interface settings which obviously is a bad idea then.


Cheers,
Franco

[Patrick M. Hausen]

While that seems clear it's also broken by design from the start because you could override the MAC address from the interface settings which obviously is a bad idea then.

Layer 2 over WAN links is broken by design. Just route, folks.

[franco]

Just to note that ZeroTier practically works flawlessly since 2018 which was the last time someone actively maintained it. But in any case more maintenance would be better... ;)

https://github.com/opnsense/core/commit/dfd9f1766d
https://github.com/opnsense/plugins/commit/4f9e03089

# opnsense-revert opnsense os-zerotier && opnsense-patch dfd9f1766d && opnsense-patch -c plugins 4f9e03089

I'm not considering hotfixing this for the same reason so much care has been taken for the initial request on the spoofmac behaviour improvements:

https://github.com/opnsense/core/issues/4430

To release this into 24.7.5 it will need a good portion of non-ZT testing as well. The normal road forward would be to include it into 24.7.6 at the earliest.


Cheers,
Franco