Zerotier stopped working after upgrade to 23.1

[atomicbargain]

I just upgraded to 23.1 and Zerotier no longer works from my router.

I have both Wireguard and Zerotier installed as separate interfaces. On 22.11 this worked flawlessly.

After the upgrade Wireguard works fine, Zerotier does not connect even though the zerotier-cli shows connected.

Something I noticed is that there is no zerotier IPv4 interface listed in the routing table and it is not listed in System -> Gateways -> Single either.

I cannot reach the Zerotier IP of the router from other Zerotier hosts.

On the Zerotier console the router shows as "up".

zerotier-cli thinks it is connected:

Code: [Select]

# zerotier-cli info
200 info xxxxxxxxxx 1.10.2 ONLINE


Plugins installed:

Code: [Select]

os-zerotier version 1.3.2_4

zerotier-cli reports

Code: [Select]

# zerotier-cli -v
1.10.2
Has anyone else seen this issue and figured out how to solve it?

[fgsfdgfds]

I'm no expert (far from it)
but after upgrade my zerotier is as expected.
local machines on opnsense side can route to zerotier clients and via versa.

Have you try a re-install?

[atomicbargain]

I have tried a reinstall, and recreated the interface with the same result.

This seems to be a routing table problem, the default root for the interface is being deleted by something in Opnsense.

My zerotier network happens to be 172.24.0.0/16

If I stop and start the zerotier service, for a few seconds I see this route via netstat -nr

Code: [Select]

172.24.0.0/16      link#11            U      zt15b2a3
172.24.15.180      link#11            UH          lo0

And everything works has expected.

Then traffic stops and this route has been removed from the routing table (I make no other changes). Only this entry is left

Code: [Select]

172.24.15.180      link#11            UH          lo0

I have a "ZERO" Interface created, and enabled. The static IP is set (following the instructions here: https://docs.opnsense.org/manual/how-tos/zerotier.html)

If I add the route manually with

Code: [Select]

route add -net 172.24.0.0/16 -interface zt15b2a3pottmrd

Then things start working again!

What would automatically reset the link routing table?

[atomicbargain]

Just to answer my own question, I resolved this.

I had the wrong configuration in 22.11 but things worked anyway. When upgrading this configuration was broken.

In the interface settings for zerotier I had set the subnet dropdown to "32". This worked in 22.11 (the zerotier program must have overridden the routing table)

I corrected this to "16" and the routing table is now correct, and traffic is flowing. I am guessing this is down to a change in how BSD manages the network stack.

[jrch2k23]

i confirm this works too, ty for your input

[drbob]

I ran in to this as well. For others just be aware that the appropriate subnet mask depends on your settings in the Zerotier control panel. If you've selected to have IPs in the 192.168 range then zerotier auto assigns IPs to other clients within a /24 by default (e.g 192.168.195.*), whilst other ranges auto-assign addresses within a /16 prefix (e.g 10.242.*.*).