After Upgrade to 24.7.4 Zerotier not working

[newsense]

Not quite there yet, the ether information is gone after the patches and the HWaddress is now displayed instead:

Pre-patches with the initial workaround

Code Select Expand

ztagimXXXXX: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 5000 mtu 2800
        description: ZeroTier (opt2)
        options=280401<RXCSUM,LRO,LINKSTATE,RXCSUM_IPV6>
        ether 3f:44:6a:e3:2c:52
        hwaddr 55:8a:dd:21:95:03
        inet 192.168.29.6 netmask 0xffffff00 broadcast 192.168.29.255
        inet6 fe80::5%ztagimXXXXX prefixlen 64 scopeid 0xd
        inet6 fca2::1 prefixlen 40
        groups: tap
        media: Ethernet 1000baseT <full-duplex>
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 54657


Post-patches and rebooted

Code Select Expand


ztagimXXXXX: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 5000 mtu 2800
        description: ZeroTier (opt2)
        options=280401<RXCSUM,LRO,LINKSTATE,RXCSUM_IPV6>
        ether 55:8a:dd:21:95:03
        inet 192.168.29.6 netmask 0xffffff00 broadcast 192.168.29.255
        inet6 fca2::1 prefixlen 40
        inet6 fe80::5%ztagimXXXXX prefixlen 64 scopeid 0xd
        groups: tap
        media: Ethernet 1000baseT <full-duplex>
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 64048

Some output has been edited of course :)

[franco]

Ok so the core patch did not apply for one reason or another... I made a backport and edited the original post.

Thanks for testing so far.


Cheers,
Franco

[newsense]

Just to confirm for everyone, patches are working now. Don't forget to restart the service :)

Code Select Expand

service zerotier restart

[Flerlato]

Ok so the core patch did not apply for one reason or another... I made a backport and edited the original post.

Thanks for testing so far.


Cheers,
Franco

Which one should we use?

opnsense-patch 1dba25fed8

OR

opnsense-revert opnsense os-zerotier && opnsense-patch dfd9f1766d && opnsense-patch -c plugins 4f9e03089

[newsense]

This one is to be used. Note that the two revert commands are only needed if the previous patches have been attempted by the people testing. If unsure it is best to run this chain of commands in full, it is absolutely safe.

I've added the last step of restarting the service to the chain as well.

Code Select Expand

opnsense-revert opnsense os-zerotier && opnsense-patch dfd9f1766d && opnsense-patch -c plugins 4f9e03089 && service zerotier restart

[franco]

My suspicion is that

# service zerotier restart

would temporarily fix this anyway?


Cheers,
Franco

[you]

Hi Franco,

concerning ... "would temporarily fix this anyway?"

I applied before mentioned patches yesterday around lunchtime. I haven't done anything since with OPNsense. And zerotier is still working as it should.

A restart of zerotier didn't do anything when the problem occurred for the first time a couple of days ago with the unpatched version.

[franco]

Ok, depending on what the zerotier binary does it may do down/up on top of fiddling with the MAC address of the interface, which would cause it to break again. It's a tough spot to be in for a VPN. ;)


Cheers,
Franco

[pkirsche]

I've added the last step of restarting the service to the chain as well.

Code Select Expand

opnsense-revert opnsense os-zerotier && opnsense-patch dfd9f1766d && opnsense-patch -c plugins 4f9e03089 && service zerotier restart

I'd like to assist with testing, but just one question before starting. If I "patch" these hotfix changes, will I need to revert these changes before or after a later released 24.7.5 or 24.7.6 including the fix or is just updating to these versions overwrite all these "patches" and nothing to do afterwards?

[newsense]

These patches will be in 24.7.5+, most likely with the latest version of ZT that was released late last week.

[franco]

I'd like to assist with testing, but just one question before starting. If I "patch" these hotfix changes, will I need to revert these changes before or after a later released 24.7.5 or 24.7.6 including the fix or is just updating to these versions overwrite all these "patches" and nothing to do afterwards?

It's complicated.

The ZT plugin change will probably stick if we update with 24.7.5 or not. The core change will be scrubbed with 24.7.5 when it isn't included (maybe because it got pushed to 24.7.6). If it's included it gets scrubbed, too, but will work regardless (because it's being included in the update).

24.7.5 is doable for next week. But I need to recheck the LAGG case and the general ability to spoof the MAC address.


Cheers,
Franco

[pranaysy]

This one is to be used. Note that the two revert commands are only needed if the previous patches have been attempted by the people testing. If unsure it is best to run this chain of commands in full, it is absolutely safe.

I've added the last step of restarting the service to the chain as well.

Code Select Expand

opnsense-revert opnsense os-zerotier && opnsense-patch dfd9f1766d && opnsense-patch -c plugins 4f9e03089 && service zerotier restart

Thanks for this! My ZeroTier mysteriously stopped working a few days ago, and I noted today that this coincided with the upgrade to 24.7.4_1. After a fair amount of trying out stuff (restarting ZT service or trying out a different network didn't work either), I had nearly given up but thankfully discovered this thread. Applying those patches as suggested fixed my ZeroTier connectivity and routing - works just like before now!

The ZT plugin change will probably stick if we update with 24.7.5 or not. The core change will be scrubbed with 24.7.5 when it isn't included (maybe because it got pushed to 24.7.6). If it's included it gets scrubbed, too, but will work regardless (because it's being included in the update).

I really hope these patches stick after those updates!

Cheers,
Pranay

[franco]

We will likely push all of them into 24.7.5. :)


Cheers,
Franco

[osmom]

Confirm, the patch works.

[Fur10us]

I can confirm that Zerotier works on a fresh install of 24.7.5.  I will update one of the previously affected 24.7.4 systems (that was working again thanks to the script posted by Newsense) tonight to verify that as well.  Zerotier was not working on multiple fresh installs of 24.7.4 (as well as upgraded units) previously.  Thank you for such a rapid solution (both the script and the fix in 24.7.5).