Adding Tailscale interface causes assignment loss

[socially.challenged.geek]

I have been looking at OPNsense for a while now and recently took the plunge into it.
To start, here is my setup.
VirtualBox VM
  4 cpu cores
  8gb ram
  2 - intel pro/1000 mt network interfaces
    1 is bridged to my laptop nic
    1 is set to host-only
  40gb qcow sata hdd
  OPNsense 24.1 installed using DVD download
  VM configured for EFI bios

After initial installation, I did the system updates and rebooted.
I did some basic poking through the web interface and I installed the virtualbox-ose-additions-nox11 package I found under firmware.
I updated the LAN address to use DHCP from the VirtualBox host-only network.
I rebooted a few times and everything appeared to be working fine.

I use Tailscale and noticed in the Tailscale documentation there is an integration for OPNsense.
https://tailscale.com/kb/1097/install-opnsense
I followed the directions... ports tree installed fine, Tailscale built fine, and Tailscale logged in fine.
I did a reboot out of curiosity to see if Tailscale would reconnect after a reboot and it did.

Now we come to the trouble part.
I went under Interfaces -> Assignments and I assigned the tailscale0 device to an interface named TS0.
Everything went fine, I went to Interfaces -> TS0 and I enabled the interface with all the default settings and IPv4 and IPv6 set to none.
Saved the changes when prompted.
Then I went to Firewall -> Rules -> TS0 and I set a simple "allow all" rule.
Everything appeared to be going smoothly... until I performed a reboot.

After performing a reboot, I was unable to access OPNsense at all.
I went to the console in the VM and logged in as root and that is when I noticed the interface assignment for WAN and LAN was wrong.
So, I pressed 1 for assign interfaces and assigned the correct devices to the correct interfaces.
After that, I had access to the Web UI again.
I logged in to discover that all of the settings pertaining to Tailscale were all gone.
No interface assigned to the device and no firewall rules.

So, I reboot without setting up any of the Tailscale stuff and the reboot is successful... everything appears fine.
I then assign the tailscale0 device to an interface and reboot.
While I am watching the console messages, I see a message that says:

Code: [Select]

Default interfaces not found -- Running interface assignment option
Press and key to start manual interface assignmentI left it timeout and notices that all the interface assignments were wrong again.

It appears that as soon as I assign the tailscale0 device to an interface, the configuration for all of the device to interfaces is lost or corrupted.

Has anyone encountered this before? I have tried to do a little searching but I have turned up empty.
Almost all of the results I find are people making the mistake of configuring a live boot environment by accident.

OPNSense looks really nice and I am excited to try migrating from PFsense but this is going to be an issue.
How can I trust that a simple change to something will not result in a loss of configuration?

I am looking forward to any help and I will do my best to provide whatever is asked of me.

[socially.challenged.geek]

I forgot to note.
After performing the updates, the version on the dashboard is now OPNsense 24.1.5_3-amd64

[socially.challenged.geek]

After walking away from this yesterday and coming back with a clear mind today, I think I might have an idea as to why this is happening.
I notice when it informs me about "Default interfaces not found..." during the boot process, if I press any key to assign devices to interfaces, the tailscale0 device is missing.
I can only conclude that, because this device is missing, the system considers the configuration of the interfaces to be corrupt or invalid and ignores loading it.

Can anyone tell me if my logic is going in the correct direction?

[RaymondFFX]

I am having the same issue. My setup is also virtualized, though through Proxmox, and just like you I followed the install instructions of Tailscale on their knowledge base that you linked to.
I had made some other configuration changes as well when I experienced this the first time but as I then changed all the settings I made since one by one with a reboot in between, I can definitely confirm this is the Tailscale interface that is the cause.

I noticed the interface has a MAC-address of 00:00:00:00:00:00 and thought this might be the issue but manually setting a MAC-address does not solve the issue.

OPNsense and Tailscale are both fully up to date, running version 24.1.6 and 1.64.1 respecitvely. Don't really know where to go from here since I'm rather new to Tailscale.

That being said, Tailscale does still work regardless of the interface being enabled or not. So I have now resorted to managing my access to the subnets through Tailscale's ACL rather than OPNsense's firewall rules but I would still like to have it as interface in OPnsense so I can manage it and firewall it like any other interface.

[socially.challenged.geek]

@RaymondFFX
Good to know I am not the only person out there with this issue. I was starting to think that was the case with all the views and no responses here.

I also noticed the MAC address of all 0's and thought about exploring that but I have not taken the time yet.
I did not think about testing if Tailscale was working or not. I did look in the Tailscale management console to see if it appeared in the list but that was all I did before moving on to other troubleshooting processes.

I have not messed with Tailscales ACL's. They appear complicated and it seems like it would require me to go make ACL changes every time I want to do something different instead of simply selecting the option in a Tailscale client. Could you please provide examples or resources on how you are creating your solution?

I am also running out of ideas on how to proceed. I am thinking about setting up a Tailscale client on the local networks that I need access to and enabling the subnet routes and/or exit nodes options. I am just really hoping I can do it all on the router so I have one device to manage and power instead of two.

I have not given up yet. If you figure out a solution, please report back.

[franco]

You should lock our WAN interface in the settings to prevent this...

This is a problem with missing integration of tailscale both in prebuilt package and GUI-based plugin. Tailscale reached out a while ago for someone to build a plugin but has not replied to our message. It sort of suggests they have other priorities so here we are.


Cheers,
Franco

[socially.challenged.geek]

You should lock our WAN interface in the settings to prevent this...

Can you explain a little about what you mean by this?

This is a problem with missing integration of tailscale both in prebuilt package and GUI-based plugin. Tailscale reached out a while ago for someone to build a plugin but has not replied to our message. It sort of suggests they have other priorities so here we are.

I kind of got that feeling from a lot of the places I have searched for help.
That is sad to hear because I have really been liking Tailscale.
There are quite a few people out there trying to get this to work.
Not only on OPNsense, but other platforms as well.
I really hope Tailscale and OPNsense can begin collaboration soon.
I think it would greatly benefit both parties as well as the communities.

[franco]

In each interface's setting there is a "lock" option near the top... it prevents the interface from being released even if it is not found on boot.

That not found on boot part is the actual issue with tailscale because it's not properly integrated and missing interface definition and hooks... which an official plugin would fix.


Cheers,
Franco

[socially.challenged.geek]

OMG!!! How did I miss that setting?
I locked the LAN and WAN, did a reboot, and things looked good.
So I went ahead and assigned the tailscale0 device to a new interface named TS0, enabled and locked the new TS0 interface, and rebooted.
All the interfaces were still there.
I went and created a firewall rule, rebooted, and everything was still there.
Thank you Franco!

This might just work.
There is no access to any of the Tailscale config in the web UI, but that is not a deal breaker for me.
Hopefully, Tailscale builds an official plugin soon. That would be perfect.

Now I will proceed with testing the setup I want to make.

[tomdh76]

I was wondering if your setup with a tailscale interface is working as you intended? I have a tailscale setup working but firewall rules are not working. I see no bytes flowing through the tailscale interface in traffic reporting. What i would like to do to is protecting the tailscale interface with zenarmor.

[franco]

The FreeBSD integration of Tailscale has a somewhat fatal flaw in that it uses a userspace network stack and you cannot witness outgoing unencrypted packets as far as I remember, but I'm not entirely sure.

The bottom line for missing traffic is that it's not seen by firewall, tcpdump or netmap.


Cheers,
Franco

[doktornotor]

The bottom line for missing traffic is that it's not seen by firewall, tcpdump or netmap.

I'd say the main problem with Tailscale on FreeBSD is here: https://github.com/tailscale/tailscale/issues/5573

There are some weird hacks with exporting TS_DEBUG_NETSTACK_SUBNETS=0 suggested, did not look into it. As it as, I'd say assigning the interfaces is absolutely pointless, these cannot be treated the usual way, firewalled via pf nor can Tailscale be used for site-to-site VPNs on BSD.

[tomdh76]

The bottom line for missing traffic is that it's not seen by firewall, tcpdump or netmap.

I'd say the main problem with Tailscale on FreeBSD is here: https://github.com/tailscale/tailscale/issues/5573

There are some weird hacks with exporting TS_DEBUG_NETSTACK_SUBNETS=0 suggested, did not look into it. As it as, I'd say assigning the interfaces is absolutely pointless, these cannot be treated the usual way, firewalled via pf nor can Tailscale be used for site-to-site VPNs on BSD.

Thx, that seems to be the issue. The work around looks difficult and hacky. Maybe i go back  to wireguard again..

[doktornotor]

The work around looks difficult and hacky. Maybe i go back  to wireguard again..

Yep. I want firewalling done via pf, this things seems to be having a parallel life mostly detached from the rest of the router. Really not for me.

[franco]

That GitHub issue sure makes it look like FreeBSD is not of any further interest to Tailscale which would also explain why we never heard back when trying to work on a plugin together.

And what I find interesting is that nobody really cares about TS_DEBUG_NETSTACK_SUBNETS=0 workaround being added to security/tailscale FreeBSD port either.

Suffice to say that people are stuck on a port that will not go forward either way. ¯\_(ツ)_/¯


Cheers,
Franco