Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - 2HgRyz13

#1
FYI -- I filed a bug report with Apple and suggested they have their macOS networking team consult their iOS networking team.

For amusement and contemplation, here are 4 routing tables (first 3 with same Wireguard config file that includes Allowed IP 10.1.10.0/24). The file server is always 10.1.10.99.

(1) WORKS: iPhone iOS 18.5 (wifi off) successful Wireguard split tunnels and file server access

Network Tools iOS app by Hurricane Electric (HE network tools):
routing table / Routes by Interface

Wireguard: utun7
Destination        Gateway      Flags
default            link#49      UCSI
10.25.25.4         10.15.25.4   UH
10.1.10            link#49      UCS
224.0.0/4          link#49      UmCSI
255.255.255.255/32 link#49      UCSI

(2) FAILS: macOS M4 Apple Silicon Sequoia 15.5, M4 Apple Silicon Tahoe Public Beta 1, Intel Sonoma 14.7.6 (no file server access; no packet flow in WireSHARK until pinging file server, which works)

netstat -rn -f inet

Wireguard: utun5
(excluding other interfaces)
Destination        Gateway      Flags
default            link#20      UCSIg (small "g" only difference with iOS's routing table)               
10.25.25.4         10.25.25.4   UH                     
10.1.10            link#20      UCS                       
224.0.0/4          link#20      UmCSI               
255.255.255.255/32 link#20      UCSI               

(3) WORKS: Windows 11 PC laptop, Windows 11 in Parallels VM in macOS Tahoe Public Beta 1 (all accessed the file server)

C:> route print

Wireguard: 10.25.25.4
(excluding other interfaces)
Active Routes:
Destination     Netmask            Interface
10.25.25.4      255.255.255.255    10.25.25.4
10.1.10.0       255.255.255.0      10.25.25.4 
10.1.10.255     255.255.255.255    10.25.25.4

Windows Wireguard uses broadcast instead of multicast. Broadcast (last octet .255 in 10.1.10.255) spews traffic to all devices in a local segment. Multicast (224.0.0/4) sends traffic to interested devices only.
   

(4) WORKS without split tunnels (Allowed IPs: 0.0.0.0/0) on all platforms tested (all accessed the file server)


Routing table from 3 macOS versions (Sonoma, Sequoia, Tahoe Public Beta 4):

Wireguard: utun5
(excluding other interfaces)
Destination        Gateway      Flags       
default            link#20      UCSg  (missing capital "I" in split tunnel routes above)       
hiding public IP   link#20      UHWIig         
224.0.0/4          link#20      UmCS  (missing capital "I" in split tunnel routes above)   
255.255.255.255/32 link#20      UCS   (missing capital "I" in split tunnel routes above)


#2
It works on my iPhone, too.

I can access the file server from iOS 18.5 with its WireGuard client using the same config file over 5G cellular (wifi off). Of all the crazy things. So just isn't working on macOS (yet).

Someone on Stack Overflow has good advice I had already tried (A,B,D not C but tried C and it didn't work). Here it is for future reference:

https://serverfault.com/questions/1189523/wireguard-vpn-split-tunnels-config-fails-from-macs-but-works-from-windows-11

Suggested Fixes

A. Add Manual Routes on macOS After WireGuard connects, manually add a route to ensure 10.1.10.0/24 goes over the tunnel:

bash Copy Edit sudo route -n add -net 10.1.10.0/24 -interface utun4 Or persist it with a post-up script (e.g., if using CLI WireGuard).

You can confirm the route is missing or misrouted by checking netstat -rn or route get 10.1.10.99


B. Use AllowedIPs = 10.1.10.0/24, 10.1.10.99/32 Some versions of macOS need very explicit IPs to enforce route handling. Add the file server's IP directly in addition to the subnet:

nginx Copy Edit AllowedIPs = 10.1.10.0/24, 10.1.10.99/32


C. Use a DNS Name for File Server (Workaround) Sometimes SMB/Finder behaves differently with DNS names than raw IPs. Try accessing:

smb://fileserver.domain.local instead of smb://10.1.10.99

This sometimes forces different resolution paths.


D. Use Full Tunnel Temporarily but Block Internet via Firewall Rule If split-tunnel is the issue, temporarily use 0.0.0.0/0 and add an outbound block rule on OpnSense for the tunnel subnet to prevent internet access.

This is a compromise until macOS routing is sorted.
#3
UPDATE

This OpnSense configuration was correct all along.

And the peer config file was correct, too. 

I dug up a Windows 11 pc, installed wireguard and imported the exact same conf file I'd been beating to death in macOS Sequoia 15.5 (Allowed IPs: 192.168.4.0/24 ).

The Windows PC WireGuard connected right away, let me connect to the file server right away, and browse the internet over the personal hotspot from my phone (as verified with whatismyipaddress).

The problem is 100% on the Mac.

M4 MBA, Sequoia 15.5
Wireguard at utun4

WireShark sees smb2 packets and more flowing on utun4 when WireGuard peer connects sans split tunnel, i.e, Allowed IPs: 0.0.0.0/0 . But with Allowed IPs: 192.168.4.0/24, WireShark shows zero packets of any type flowing on utun4 until I ping the file server.

On the M4 MBP Sequoia, I tried...
... wireguard gui client (same problem)
... passepartout macos wireguard client (same problem)
... defguard client (free) (same problem)
... homebrew version of wireguard and tested it in CLI (same problem)

On a stripped down old Intel macOS Sonoma 14.76 tried ...
... wireguard gui client (same problem)

I found other users complaining about the same or similar problem with split tunnels on macOS:
"MacOS Client does not create valid Routes" by a network engineer
https://github.com/DefGuard/client/issues/281

Though that's post is about DefGuard/WireGuard, it seems like macOS routes aren't correct.

Again, the exact same WireGuard conf file worked in Windows 11 WireGuard client.

After connecting macOS Wireguard GUI client, running
netstat -rn -f inet (-f inet for ip4 only) shows the macos routes

ifconfig shows the utun4 after Wireguard starts.

Ok, so for now, I'm taken my troubleshooting to the Wireguard support IRC since the problem is definitely NOT an OpnSense problem.

If I figure it out with Wireguard support, I post the solution here. 

Oh, and I upgraded OpnSense to 25.7 before I realized the problem is 100% on the Mac.

#4
Quote from: Patrick M. Hausen on July 22, 2025, 04:23:03 PM
Quote from: 2HgRyz13 on July 22, 2025, 04:10:38 PMWhen I clicked "Apply" all hell broke loose. The peer page in the GUI became unresponsive, then the list of my numerous peers disappeared!!
[...]
Wildddd bug.

Nope. Not a bug. You explicitly told OPNsense to route 10.1.10.0/24 into the WireGuard tunnel instead of your LAN thereby losing connection to your LAN. Don't do that ;-)

BTW: your settings look ok now. If it still does not work, check firewall rules and as a last resort use tcpdump to trace packets as they (hopefully) travel through the tunnel.

BTW2: you cannot test from inside LAN with your Mac. You need an external Internet connection. Same reason as your almost desaster above - the Mac cannot be connected to 10.1.10.0/24 via it's local network and via the WG tunnel at the same time.

Yes, I know, I've been testing the Wireguard connection through a Personal Hotspot. I verify I'm on an external network by visiting whatismyipaddress on the client computer. It shows the IP is my cellular provider's instead of the gigabit fiber into the DEC 850.

I frequently retest the client with Allowed IP 0.0.0.0/0 just to make sure I can reach the file server ok. It always works.

Then I went back to trying different Allowed IP setting in the client conf file: 10.1.10.0/24 and 10.1.10.99/32.

I need to stop for now because I haven't slept for a long time and we all know that can be dangerous.

Thank you very much for your detailed explanations and confirming my settings look ok!
#5
Quote from: 2HgRyz13 on July 22, 2025, 04:13:23 PM
Quote from: Patrick M. Hausen on July 22, 2025, 02:17:18 PMOn the peer (Mac) I have AllowedIPs = 192.168.0.0/16 - which means that traffic to this entire network goes into the tunnel from the Mac to the OPNsense.

On the OPNsense side I have AllowedIPs = 192.168.255.2/32 which means that the OPNsense sends traffic to this single address into the tunnel.


Really in WireGuard there are no clients and servers, just peers. AllowedIPs tells WireGuard what is supposed to be at the other side of the tunnel.

That's why you put 0.0.0.0/0 on your "client" into AllowedIPs when you want to tunnel everything, and only select networks when you want split tunnel. I put "client" into quotation marks because it's really just an administrative distinction - the always on system waiting for incoming connections (OPNsense) vs. the road warrior initiating a connection (Mac). For WireGuard there is no difference.

Repeating myself: AllowedIPs is what is at the other end.

Thanks again. I think I have a better understanding of the terminology now but will think about it more later.

Do my settings seem ok to you?


Yes, the Allowed IPs are what is at the other end. The Peer Generator did it correctly before I understood it from your explanation. The AllowedIP in the OpnSense peer edit-view shows that it allows that one IP address used by the client computer's Wireguard VPN, and the client computer's config's Allowed IP shows the network range on the other end (my LAN) that's allowed, 10.1.10.0/24.

I think I'm getting it.
#6
Quote from: Patrick M. Hausen on July 22, 2025, 02:17:18 PMOn the peer (Mac) I have AllowedIPs = 192.168.0.0/16 - which means that traffic to this entire network goes into the tunnel from the Mac to the OPNsense.

On the OPNsense side I have AllowedIPs = 192.168.255.2/32 which means that the OPNsense sends traffic to this single address into the tunnel.


Really in WireGuard there are no clients and servers, just peers. AllowedIPs tells WireGuard what is supposed to be at the other side of the tunnel.

That's why you put 0.0.0.0/0 on your "client" into AllowedIPs when you want to tunnel everything, and only select networks when you want split tunnel. I put "client" into quotation marks because it's really just an administrative distinction - the always on system waiting for incoming connections (OPNsense) vs. the road warrior initiating a connection (Mac). For WireGuard there is no difference.

Repeating myself: AllowedIPs is what is at the other end.

Thanks again. I think I have a better understanding of the terminology now but will think about it more later.

Do my settings seem ok to you?
#7
So here's the insane thing that happened while I investigated the Allowed IPs in the created peer in OpnSense.

When I went into the edit view in OpnSense of an existing peer to view its Allowed IPs, I saw just the IP address that the vpn client would get when it connected 10.25.25.4/32.

For the heck of it, because nothing else worked so far, I clicked in the field next to that address and typed in the Allowed IP range that should only be in the client's config, 10.1.10.0/24. Now there were two IPs in that field.

I clicked Save.

When I clicked "Apply" all hell broke loose. The peer page in the GUI became unresponsive, then the list of my numerous peers disappeared!! then reloading the page in the browser didn't work, then I couldn't reach OpnSense, and the internet was down. I power cycled the DEC 850 OpnSense and waited. Didn't help, still couldn't reach it in a browser. I tried changing my computer's IP address to match the default in a new DEC 850, 192.168.1.1, but still couldn't reach. I switch the computer back to its regular IP address but still couldn't reach the OpnSense.

I used my Mac to connect to its console (ls /dev/ then screen /dev/ttyusbblahblah... , then my root password in my password manager didn't work nor my admin credentials, so booted the DEC 850 again (power cycle) in the OpnSense menu and choose Single User mode, then reset the root password, then restarted again and restored the config from the menu by choosing a backup from about 6 hours ago.

Wildddd bug.

I'll report it later.

DON'T TRY IT UNLESS YOU'RE READY FOR DISASTER.

Fortunately, it was the middle of the night here and users won't know. I've been up all night wrestling with split tunnels and that was just the cheery on top. HA HA. Hold my calls. I'll go to bed soon.

#8
Ok, thanks.

I see what you mean now.

What confused me, among other things, is that the Peer Generator created both the edit-view peer you showed in your OpnSense plus it created the peer settings for the client (for me via the QR code).

So the client's config is what I've been working with, its Allowed IPs: setting, which in my case in my LAN net expressed as 10.1.10.0/24 (in your client config its 192.168.0.0/16).

When I went into the edit view of the instance, I see the equivalent of what you showed on yours, a tunnel address of 10.25.25.1/24.

And the edit view of the already-created peer (as opposed to just the Peer Generator view during peer creation), also shows the equivalent of yours, 10.25.25.4/32.

All without the IPv6 addresses in my case.

The Peer Generator created the peer properly according to your screenshots.

And the Peer Generator created a QR code I used for the client, after which I manually viewed and edited that config file on the client: Allowed IPs: 10.1.10.0/24 (but I also tried 10.1.10.99/32 and that with the first separated by a comma). The file server is at 10.1.10.99 btw.

So unless I'm misunderstanding, my instance, the created peer listed in OpnSense and peer config in my Mac client seem correct.

Thanks for pointing out that a created peer in edit view in OpnSense only reveals its Allowed IPs: of the the client's IP, whereas the client config on the client computer is the only place listing the actual LAN network range in its Allowed IPs field.

Whew.

I'm going to add something wild in the next post instead of at the bottom of this one.
#9
Quote from: Patrick M. Hausen on July 22, 2025, 01:54:55 PM
Quote from: 2HgRyz13 on July 22, 2025, 01:28:04 PMThere are several WireGuard clients available for macOS, including the official WireGuard client and third-party alternatives. The official WireGuard client is available on the Mac App Store and provides basic functionality for managing and using WireGuard tunnels. However, it lacks advanced features such as split tunneling and more access to configuration options.

That's plain wrong. The official Mac client takes a text based configuration with all options you can place in that text field.

My client with split tunnel looks like this:

[Interface]
PrivateKey = ****
Address = 192.168.255.2/32, 2003:a:d59:38ff::2/128
DNS = 192.168.255.1, ettlingen.hausen.com, hausen.com
MTU = 1280

[Peer]
PublicKey = ****
PresharedKey = ****
AllowedIPs = 192.168.0.0/16, 2003:a:d59:3800::/56, 2003:a:d7f:d938::/64
Endpoint = ****:51820
PersistentKeepalive = 25

On the OPNsense side this is the instance - you can ignore IPv6 if you do not use it:



And this is the corresponding peer - it has the peer's interface address in AllowedIPs, necessarily so:



HTH,
Patrick

I'm confused.

Your config files shows
AllowedIPs = 192.168.0.0/16,...

Yet your screenshots of the Peer client don't that, just 192.168.255.2/32 (...) which is the first ip address of the IP address the instance gives out from its tunnel address def of 192.168.255.1/24.

Your screenshot examples are radically different from all other documentation I've found. But what the heck. I'll try it.
#10
Quote from: 2HgRyz13 on July 22, 2025, 01:28:04 PMDang, I just read this in a Brave AI search result about Wireguard clients for MacOS.

There are several WireGuard clients available for macOS, including the official WireGuard client and third-party alternatives. The official WireGuard client is available on the Mac App Store and provides basic functionality for managing and using WireGuard tunnels. However, it lacks advanced features such as split tunneling and more access to configuration options.

But it doesn't show a source for that, just for sentences preceding it.

Still, this would explain it.

I'm going to try at least one other macOS Wireguard client.

Until I do, I don't recommend trying to help me with this until I report back, except please tell me with macOS Wireguard client you like.

I just tried Passepartout app from the app store. Highly rated. Same problem, though.

Wireguard 1.0.16 (27) - experiences the issue
Passepartout 3.5.4 (3880) - experiences the issue

Because both experience issues, I think the problem is with OpnSense 25.1.11, macOS 15.5 or me.

I also doubt Brave AI's comment that Wireguard's own macOS app doesn't handle split tunnels. It has the Allowed IPs field and it's a basic feature. Unless someone knows otherwise.



#11
Quote from: franco on July 22, 2025, 01:28:53 PMApart from https://github.com/opnsense/core/issues/8974 when using default routes from wireguard (or other VPNs) consider using 0.0.0.0/1,128.0.0.0/1 as a default policy to avoid clobbering your system's default route 0.0.0.0/0.


Cheers,
Franco

I don't plan on using 0.0.0.0/0 except for testing. My goal is to use a split tunnel 10.1.10.0/24

#12
Dang, I just read this in a Brave AI search result about Wireguard clients for MacOS.

There are several WireGuard clients available for macOS, including the official WireGuard client and third-party alternatives. The official WireGuard client is available on the Mac App Store and provides basic functionality for managing and using WireGuard tunnels. However, it lacks advanced features such as split tunneling and more access to configuration options.

But it doesn't show a source for that, just for sentences preceding it.

Still, this would explain it.

I'm going to try at least one other macOS Wireguard client.

Until I do, I don't recommend trying to help me with this until I report back, except please tell me with macOS Wireguard client you like.
#13
Quote from: Patrick M. Hausen on July 22, 2025, 01:09:20 PMWhat's the AllowedIPs setting for the peer on the OPNsense side?

Not sure what you mean.

I thought the Allowed IP setting is only on the client side, though I used the OpnSense firewall's built-in Wireguard Peer Generator to create the peer with Allowed IP= 10.1.10.0/24. Then I continued trying Allowed IP values with manual changes to the config file on the client Mac.

OpnSense Wireguard Instances don't have an Allowed IP setting, just the tunnel address setting, e.g., 10.25.25.1/24.

The file server is at 10.1.10.99, which is reachable through non-split-tunnel connections (Allowed IPs: 0.0.0.0/0, ::/0).
#14
Here we go...

[Interface]
PrivateKey = redactedxxxxxxx
Address = 10.25.25.4/32

[Peer]
PublicKey = redactedxxxxxxx
PresharedKey = redactedxxxxxxx
AllowedIPs = 10.1.10.0/24
Endpoint = redactedxxxxxxx.redactedxxxxxxx.com:51222



The file server is on the LAN at 10.1.10.99.

Non-split tunnel clients successfully access the file server through a firewall rule that only allows access to 10.1.10.99 on the LAN, but the access only works with a client using Allowed IPs: 0.0.0.0/0, ::/0 (not split tunneled)

I also tried MTU = 1280 (didn't help)

And I tried Allowed IPs:
       10.1.10.99/32
then 10.1.10.0/24, 10.1.10.99/32
then 10.1.10.0/24, 10.1.10.99/32, 10.25.25.0/24

None helped. Split Tunnels sure look easy to set up, with just the Allowed IP = the LAN net x.x.x.0/24.

DNS shouldn't matter since we access the fileserver by IP address, not by name, but I tried the following anyway in the config (none helped)
        DNS = 1.1.1.1
then DNS = 10.1.10.1
then DNS = 10.1.10.1, 10.25.25.1
then DNS = 10.1.10.1, 10.25.25.1, 1.1.1.1

Disabling Unbound DNS didn't help (as expected)
Disabling Outbound NAT & Normalization didn't help (as expected)

I suspect a bug in...

• OpnSense 25.1.11
• or my macOS WireGuard client 1.0.16 (27)
• or macOS Sequoia 15.5

What macOS Wireguard client do you use successfully with split tunnels?

I just read a notice that 25.1.12 is released, but my DEC 850 doesn't find it yet. Maybe in hours.
#15
Quote from: Patrick M. Hausen on July 22, 2025, 12:00:44 PMTry setting mtu 1280 at the peer.

Thanks for the idea. I just tried it but it didn't help.