After Upgrade to 24.7.4 Zerotier not working

[Mann-IT]

HI,

after upgrade to 24.7.4 my zerotier connection are death.
Move back to 24.7.3 they are working.
What is wrong?

Greetings Mario

[dalton_lopes]

here stops too after updates.
I migrated to openvpn in a hurry but i like to use zerotier.

[bankja]

Same here. Reverted back to 24.7.3-1 for now.

[dalton_lopes]

how rollback? how do it remotely?

[franco]

This seems to be a common theme but it's a wild situation.. which change, which component update of this affects ZeroTier operation?


Cheers,
Franco

[pbk]

There have been issues in the past, mostly because of routing issues within OPNsense – I guess – in all cases the device was connected properly to ZT was able to see all neighbours. But no comms to OPNsense. A zt leave and zt join fixed it (or removing the checkmark for the network in the gui).

This time it's different. Everything works with 24.7.3_1 (and with many releases prior, too. The above issue was last seen sometimes earlier this year). After installing 24.7.4 ZT no longer works.

A zt leave and zt join don't solve it anymore, no comms.

The firewall rules to allow data flow from the ZT network to other networks or the firewall itself don't show any states in inspect mode.

Reverting to 24.7.3_1 restores ZT connectivity immediately.

Question is: how to hunt down "which change, which component update of this affects ZeroTier operation"? Any directions on where to start?

[newsense]

=======
Initially I thought it was in the kernel - reverted the kernel to 24.7.3 - no dice.
=======
Then I reverted the OPNsense package - restarted ZT - success.
   - kernel and OPNsense on 24.7.3 - the rest on 24.7.4
=======
Reinstalled 24.7.4 kernel - so fully on 24.7.4 except OPNsense which is on 24.7.3 - success again.
=======


With the kernel out of the way - which was the only sane option - best guess now is this is caused by PHP (?)

[franco]

> Reverting to 24.7.3_1 restores ZT connectivity immediately.

Someone will tell me what this means? Full revert? Still leaves the question if this is a kernel or core issue...


Cheers,
Franco

[franco]

Ok so it's core but that still leaves a number of guesses:

https://github.com/opnsense/changelog/blob/ffb1c305508e360a4bcaa131e562d56d393ef2b4/community/24.7/24.7.4#L27-L53

It would probably be best if someone with the issue could do a bisect on stable/24.7 between tags 24.7.3 (good) and 24.7.4 (bad).


Cheers,
Franco

[pbk]

> Reverting to 24.7.3_1 restores ZT connectivity immediately.

Someone will tell me what this means? Full revert? Still leaves the question if this is a kernel or core issue...

It means the whole system was reverted to 24.7.3_1.

[Cerberus]

Same issue on my end.

I run a Zerotier Tunnel between a OPNsense Business Edition (home) and OPNsense Community running at my hoster. Right after updating to 24.7.4 on the OPNsense Community Edition, Zerotier is dead. Both Zerotier installations are shown as online, but none of the devices can ping each other on their Zerotier IP or any other IP that is routed over this Tunnel.

I try to downgrade my OPNsense to 24.7.3 as a solution for now.

Update: Downgrade with "opnsense-revert -r 27.7.3_1" worked, traffic is fliowing again

[franco]

Since nobody helped further so far my best guess is https://github.com/opnsense/core/commit/1dba25fed8 and someone will need to confirm or deny that's the one. I'm assuming we're talking about assigned ZeroTier interfaces? At first glance this has nothing to do with ZeroTier...


Cheers,
Franco

[Mann-IT]

HI,

sorry I do not unterstand what do to?


Greetings Mario

[franco]

# opnsense-patch 1dba25fed8

Doesn't matter if 24.7.3 or 24.7.4.

If applied to 24.7.3 it should get worse. If applied to 24.7.4 it should be better -- given that it's the bad commit in question.

What I'm trying to tell here is that nothing related to ZT was changed so it's unclear what the problem is.

First we find out which commit. Then we need to figure out why ZT doesn't like it. Someone with the problem needs to do this.


Cheers,
Franco

[Patrick M. Hausen]