20.7 Zerotier one - poor client performance

[curto]

Guys,

Migrating a client away from Cyberoam to OpnSense. First foray into using it - so far very impressed.

One issue i have

Gigabit link to Internet
Stock install - no IPS, IDS or any apps added.
Whilst in testing mode we have allowed all outbound ports.

Outbound clients to internet (Speedtest.net) are achieving 900+Mb/s both upload and download - so very happy with that.

We have a number of machines behind the firewall with Zerotier client installed.  We RDP across the ZT network onto those machines and this is where the problem is.

I will concentrate on a single machine (but it is happening across all of the ZT machines that are accessed through RDP.

The machine i access for management purposes is a W2K8 server  - i have created a Management ZT network that i am able to access for this site and there is a machine on another site on the same ZT network (no other VPN between sites)

So we have 3 machines on 3 different sites on the same ZT subnet.

On the problem site we are in the process of migrating away from Cyberoam SG series firewalls. Previously we were using ZT fine with these in place and speed was excellent.

Now that we have changed FWs we are finding the interactive speed it terrible - when pinging one of the machine on the 3rd site that is still behind a Cyberoam firewall i am getting 12ms response times, on the one with OpnSense i am getting 450ms.

If i swap back to the Cyberoam from OpnSense (even though it is not fast enough to handle the Gigiabit speeds) the response times to pings to the problem machine drop back to 12 to 13 ms.

Both remote machines are located relatively close to me in Sydney - and both are connected to the same ISP network - the network is a dedicated business grade fibre network - so the performance issues are not coming from there.

Has any one else experienced problems with ZT clients behind OpnSense ?

It feels to me like MTU fragmentation but i have left that at the default settings.

The firewall is an Intel i5 with 6 gigabit network interfaces (one of the QOTOM units) - it has a samsung SATA SSD and 8GB RAM.

ANy ideas ?

Craig

[Cerberus]

This high pings really looks like your are using the zerotiers own fallback gateways, they are slow and laggy.

If you use Zerotier on Opnsense, make sure you open 9993/UDP on the WAN Side, if you run Zerotier clients on the internal network, enable Opnsense UPNP Plugin or Zerotier is unable to open dynamic ports and fall back to Zerotier public fallback gateways, this especially happen if all clients are behind nat and cannot find a way to build up a direct connection / NAT workaround.

[curto]

Thanks Cerberus.

Nope i have not installed ZT on OpnSense (thats probably the next step) - we have clients (Virtual Desktops) behind OpnSense that have ZT installed.

Remote users are made members of the ZT network so they can RDP into the Virtual Desktops.

When you say enable a rule on the External interface for UDP port 9993 would that look something like

Enable incoming on Wan, from any IP address, source UDP 9993, Destination  internal LAN network ?

Thanks for taking the time to answer

Craig

[curto]

Thanks for the help - it was the upnp that did it.

Even though it was installed (must be part of the base install) it was not showing up in the services, I had to uninstall the plugin, then reinstall and it showed up. Once configure ZT when down to 11ms pings - so all good.

Craig