Odd ZeroTier behavior 24.7.5_3

Started by cakeofhonor, October 01, 2024, 08:31:00 AM

Previous topic - Next topic
October 01, 2024, 08:31:00 AM Last Edit: October 01, 2024, 08:37:40 AM by cakeofhonor
I was following this thread: https://forum.opnsense.org/index.php?topic=42798.0, and finally upgraded when a couple of people confirmed ZeroTier to be working on 24.7.5. However, I'm experiencing an odd behavior that I can't figure out.

OPNsense and my laptop are both connected to ZeroTier via managed routing. Ping and SSH both work, but trying to access any service from the web browser doesn't.

I see my in rule allowing Zerotier to pass to LAN:
ZeroTier tcp pass ZeroTier network to LAN network

Followed by the default out rule with the same destination / source address and ports:
LAN tcp let out anything from firewall host itself

But my browser just keeps loading until it eventually times out.

I also have a WAN rule that allows tcp/udp on port 9993 to This Firewall on OPNsense.

Other strange things that I've noticed are:
- I can ping but not ssh or traceroute to the OPNsense box and can't access the web GUI.
- I can ping, ssh, and traceroute to all other devices on LAN, but can't access any services in the browser (timeout).
- Upon restart I noticed that connections via tcp to any port from the browser on my laptop are blocked by the "Default deny / state violation rule", but my connections via ssh to port 22 passes. The browser connections start passing after a minute or two but the browser will still timeout.

I am using this in conjunction with WireGuard, https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html, and am behind a CGNAT, but this setup was working perfectly before 24.7.4.

So I was able to figure out why the packets were being dropped. It has something to do with the MTU, so I just set the MTU on the ZeroTier interface to 1280 (I've tried slightly higher but that doesn't seem to work) and everything started working, but the speed isn't great.

I'm not exactly sure what MTU does, so if someone can comment on why the higher default MTU (2800 for ZeroTier) causes packets to be dropped, I'd love to know.