Tailscale - Site-to-Site Subnet routing

[nsmith17044]

Has anyone successfully installed Tailscale on two instances of OPNSense and got sub-net routing to work/route between OPNSense tailnet nodes? 

I'm running V25.7.10 with Tailscale Plugin 1.3 / Tailscale 1.92.2

I've been working on this for a couple weeks and just can't seem to move forward.  Subnets are IPv4 and WAN is CGNAT fixed wireless on both ends.  Individual nodes on the tailnet can connect to machines on each OPNSense network and visa-versa so I know that the basic sub-net routing IN is working but I have been unsuccessful at getting the tcp/udp traffic to route between the subnets.  Ping between OPNSense instances does work but no tcp/udp sessions.  I suspected this could be a NAT issue but i'm struggling to understand how to diagnose the problem and fix it.  My experimentation with NAT rules has only broken what was working so I keep going back to the initial Tailscale configuration for OPNSense.  Packet capture around the tailscale interface hasn't given me anything as I only see the initiation of a request on the LAN and a bunch of the tailscale wiregaurd protocol packets on the WAN interface.  I'm also struggling to see any firewall messages where something is blocked in or out.  My tailnet routing grants are still the default src

• , dst
• so I'm expecting everything to be routing right now.  Interestingly when I specified the specific sub-nets in tailscale JSON per the tailscale documentation on sub-net routing nothing routes in from individual tailnet nodes. 

Anyone know how to troubleshoot this or have a guide on how to set this up? 

[eimann]

I've got the same issue, unfortunately without a solution.
Tailscale ACL works, traffic also works on Linux (with disabled auto-SNAT as I want to preserve source IP).
However, when deploying on OPNsense, it breaks.

ping von freeradius => wlc tut (sehe icmp auf dem ts-sidecar in beide richtungen, auf der opnsense tailscale0 gar nix, auf dem lan interface aber schon)
ping von wlc => freeradius nicht, sehe icmp auf der opnsense tailscale0 ausgehen, auf ts-sidecar eingehend + ausgehend

ping from LAN A to LAN B
 traffic outgoing on OPNsense TS interface => incoming+outgoing on TS other subnet router <=> incoming+outgoing on LAN other Subnet Router

ping from LAN B to LAN A
 incoming on LAN other subnet router => outgoing on TS interface other Subnet Router => traffic NOT incoming on OPNsense TS interface

Firewall rules permit everything between these hosts/subnets. And of course with NAT it works, but as said before, losing source IPs which I need.

[nsmith17044]

@eimann: Seems that maybe you have made it a little further than I have.  Are you indicating that LAN A to LAN B tcp/udp traffic is working or just ICMP?  At one point I saw ICMP working but some how I no longer see that on my end.

My NAT outbound are currently set to Automatic / no manual rules for the TS interfaces.  I have one rule on the tailscale interface on each to accept any/all IPv4 traffic.

Can anyone make a recommendation on what diagnostic features might be helpful to see what is happening with Tailscale?

[eimann]

When pinging from the host (A) behind OPNsense to the Host (B) behind the Linux TS subnet router, I receive the traffic on host B, it replies and I see it leaving the Linux TS subnet router. However, I do not see the reply on the OPNsense TS interface, therefor never reaching host A.

When pinging from the host (B) behind Linux to the Host (A) behind the OPNsense, I do not receive any traffic on the OPNsense.

[nsmith17044]

After doing quite a bit of troubleshooting it appears that the issue is with Tailscale on FreeBSD.  Is is NAT'ing the tailnet traffic ("NoSNAT: false") and I cannot find a way to disable it.  It seem to be a built in setting maybe?

At this point this seems to make sub-net to sub-net routing with IP preservation impossible over a Tailnet with OPNSense.  Seems like a really common thing to do but this doesn't make any sense that it isn't supported.

Seems that I would need to setup another Linux system behind OPNSense just to run Tailscale for sub-net routing on each network I need to connect to.

Anyone have a proven way around this with OPNSense?   

[nsmith17044]

As an update to my recent post the AI support agent at Tailscale confirmed the subnet routing NAT behavior is enabled by default in the FreeBSD/OPNSense build and there is no plan/schedule to fix this.

The AI agent explicitly recommended using a Linux subnet router behind OPNSense running Tailscale to perform subnet routing. 

The information I've been finding on Zerotier seems to infer it isn't primetime for fixed wireless / CGNAT configurations.  Does anyone have anything good to say about Zerotier on OPNSense when on fixed wireless with CGNAT and performing subnet routing? 

[nsmith17044]

I spent the last few hours trying to setup a new ZeroTier net and it seems they have taken a few steps backwards in sub-net routing with FreeBSD/OPNSense.  The new portal doesn't even expose the options for subnet routing with FreeBSD nodes.  CoPilot had me hacking the ZeroTier local.conf to advertise the subnets / routes but I couldn't get any of that to allow packets across the OPNSense instances.  The new portal doesn't seem to allow you to get an API token for the ZeroTier config panel so I couldn't use that.  I did notice that on the public side I wasn't playing the IP address lottery so ZeroTier's keep awake signalling must have improved somewhere along the way.

Feels like TailScale with a standalone Linux VM doing subnet routing as the only option left that seems viable.