OPNsense Forum
Archive => 23.1 Legacy Series => Topic started by: RamSense on January 27, 2023, 08:39:36 am
-
Anybody else having problems with Wireguard Kernel vs go?
Wireguard road warrior, always vpn running on devices like Iphone etc. So when back at home at wifi, the devices are still using vpn (just laziness and having vpn while leaving wifi). But with the kernel version now with Opnsense OPNsense 23.1-amd64, after some time the device (on wifi with vpn) does not get any data, no browsing, no apps, looks like traffic gets blocked. rebooting the opnsense box fixed it, but again after some time, same bug.
I swiched back to the *go version now, and all is working like it should. I do not know how to find the log or seek errors, that is why i mention it here so others can test and see if they can replicate it.
I'm using wireguard with ipv4 and ipv6
-
Switch back to os-wireguard-go if you have issues.
I do still consider wireguard-kmod an elaborate hit-and-run and that issues remaining will only be gradually improved as new FreeBSD versions are being released...
Cheers,
Franco
-
franco, I have no problems with either - go or kmod. What's the official default configuration now for 23.1? When using the kmod, the services widget still lists wireguard-go and flags it red.
Thanks! Perfectly smooth upgrade so far.
Patrick
-
kmod is the default now because users have been pushing for it. It will also be available in FreeBSD 13.2 as far as I understand. But there are hiccups with it for sure as we can see now with 23.1. Minor ones, but disruptive nonetheless.
I made a patch for the service widget... https://github.com/opnsense/plugins/commit/2ed1f987eb97d
# opnsense-patch -c plugins 2ed1f987eb97d
Cheers,
Franco
-
Excellent! Came here to talk about the service widget indeed. Will try this patch :)
Edit: And the patch works perfectly! Thanks :)
-
Same - works. Thanks.
-
Yay, progress :)
-
Thanks, patch is working.
Do I have to worry with updates in the future after patching? Like, do I have to do anything with the next update or will all things be handled automatically when this is updated in the main version?
-
Will be included in 23.1.1 so no problem with update.
Cheers,
Franco
-
Thanks all for the replies and update/patch. I will test it again when version 23.1.1 is available.
-
What exactly is "it"? I don't expect functional changes for either go or kmod in 23.1.1.
Michael mentioned a netmask issue between go and kmod where kmod is more restrictive and only allows /32 endpoints?
Cheers,
Franco
-
I have 2 endpoints configured.
1 ipv4 /32 and 1 ipv6 /128
the "it" is what I can not identify (yet). The kernel version works at start, but after some time stops working. It could be to do with the ipv6 endpoint. The go version keeps on running as it should.
-
Ok, but in this case I have no hopes for 23.1.1 from what we know today. Something will be wrong somewhere, but any other 23.1.x may be more realistic.
I'll try to place this here again for emphasis:
kmod is the default now because users have been pushing for it. It will also be available in FreeBSD 13.2 as far as I understand. But there are hiccups with it for sure as we can see now with 23.1. Minor ones, but disruptive nonetheless.
Cheers,
Franco
-
Thank you Franco, The patch worked.
Al
-
@RamSense what's the time we are talking about?
I have also set up my clients using v4 and v6. Normally my Android phone automatically disconnects WG when at home wifi, but for testing purposes I disabled this behaviour and for now I am online with WG on wifi coming home with WG on 5G/LTE for about one hour without any problems.
-
@tiermutter thnx for testing! I have not timed it exactly, but it was after several hours. If i head to guess I cap it at 4 hours. Would be great to hear back if it gets broken at your end also, or keeps on working.
-
Ok, I'll keep on testing. WG is now up since about 02:30 pm.
Do you connect to WG server with ipv4 or v6? I am on v6.
-
mhhh... looks like there was an issue shortly before I sent my last post, but it seems to be "my" fault:
For a short time I stayed on a place where wifi signal is not very good and WG stopped saying
Failed to send data packet: write udp6 [::]:53889->[2a00:xxxx:xxxx:xxxx::xxxx]:55190: sendto: network is unreachable ::)
Now tunnel is up since 04:28 pm..... now I also activated the tunnel on my PC.
-
thanks for the update. I have my clients config set with both ipv6 and ipv4 addresses. I think that the ipv6 get's picked by preference or because it is stated as first and ipv4 as second address.
To make it even more weird, at the time of error, some iphone's with wg and wifi where still working, and others were not, and after the opnsense reboot, and second failure moment another device what was working the first time, was now also failing.
where do you find the log infoFailed to send data packet: write udp6 [::]:53889->[2a00:xxxx:xxxx:xxxx::xxxx]:55190: sendto: network is unreachable
?
-
I have my clients config set with both ipv6 and ipv4 addresses.
Yes, v4 and v6 inside the tunnel... but how do you estabish a connection to the WG server?
where do you find the log info
It's on Android/ Settings (three dots upper right).
-
but how do you estabish a connection to the WG server
by my own domainname referring back to opnsense with it's ipv6 and ipv4 ip.
It's on Android/ Settings (three dots upper right)
ah ok, i'm on iphone and the wireguard app log is not going back long enough to go back tot the error moment. I will keep that in mind when i'll go for a retest at some point.
-
by my own domainname referring back to opnsense with it's ipv6 and ipv4 ip.
Ah ok, sorry, now I understand :)
-
Six hours later nothing happend, everything is working fine, Windows and Android client are still connected via IPv6 GUA. Watching the WG server status I remember that my NAS is always connected as client via IPv6 ULA. Since update / reboot it's about 8:45 hours.
-
thanks for the update and reporting back! I have just installed the kernel version again. After installation I noticed that the service widget was still working, while with the initial update of opnsense it was not / showing red. While the wireguard go version gets removed, I may assume that directly after the install of the kernel version all connected vpn devices are connected by the kernel version and not the removed go version.
Nevertheless, I installed the opnsense-patch -c plugins 2ed1f987eb97d right after installing wireguard kernel.
I will test it now and see what happens. When something happens I will try to find something in the log.
Will report back
p.s. have you also tested if wireguard kernel keeps on working after rebooting the opnsense box?
-
The patch makes the service widget show the correct status of WG kmod, that should have nothing to do with your problems I guess.
No, havn't rebooted yet, maybe late this evening when I am calibrating my UPS after battery change...
-
The patch makes the service widget show the correct status of WG kmod, that should have nothing to do with your problems I guess.
Exactly, that is what i meant. The service widget was already showing the correct status after installing wireguard kernal (what deinstalled the go version automatically), and before i installed the patch. That was the first different behavior to the upgrade of opnsense. With the opnsense update and installed wireguard kernel, the service widget was not working / showing wireguard as down, while the kernel version was running.
I'm very curious what happens after your reboot, read in another thread of the wg interface being reported down (wg0) at boot https://forum.opnsense.org/index.php?topic=31889.msg155319#msg155319 (https://forum.opnsense.org/index.php?topic=31889.msg155319#msg155319)
-
It has been almost 4 hours and still all systems go.... Makes me wonder about 2 things:
1. Does your wireguard kernel keeps working after a reboot?
2. Can it be that the opnsense update did something different than the manual installation of wireguard kernel (with auto removing wireguard go) ?
-
Did a reboot and again everything is working fine :)
Don't know if there is a diference between new install and update...
-
I'm experiencing a similar issue after upgrading to 23.1 where Wireguard handshakes are timing out when at home and decided to do some debugging.
My Android Wireguard client is setup pointing at a hostname, vpn.mydomain.com:51820, which is an A record pointing at my public IP, and I use the Always-on VPN feature in Android on this tunnel. I have all 3 of the NAT Reflection settings in OPNsense's settings (under Firewall > Settings > Advanced) turned on.
igb1 is my LAN, igb2 is my WAN, and wg1 is the Wireguard interface. When I caught the Android client sending handshakes and timing out, I turned on debugging for wg1 (ifconfig wg1 debug) which showed that OPNsense was receiving the handshake and sending a reply to the client, which led to me dig deeper.
wg1: Receiving handshake initiation from peer 1
wg1: Sending handshake response to peer 1
wg1: Receiving handshake initiation from peer 1
wg1: Sending handshake response to peer 1
I checked tcpdump on igb1 and I was able to see the handshake packets from my phone (192.168.1.68) directed to my public IP (let's call it 203.0.113.7), however there was no traffic flowing back to the phone:
root@opnsense:~ # tcpdump -nn -i igb1 host 192.168.1.68 and port 51820
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb1, link-type EN10MB (Ethernet), capture size 262144 bytes
10:45:49.935640 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
10:45:54.967559 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
10:46:03.355900 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
10:46:11.883729 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
I then checked igb2 and noticed that it is sending the traffic destined for my LAN out the WAN interface:
root@opnsense:~ # tcpdump -nn -i igb2 host 192.168.1.68 and port 51820
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb2, link-type EN10MB (Ethernet), capture size 262144 bytes
11:14:38.316335 IP 203.0.113.7.51820 > 192.168.1.68.35190: UDP, length 92
11:14:43.354698 IP 203.0.113.7.51820 > 192.168.1.68.35190: UDP, length 92
It seems that after some indeterminate period of time, wireguard-kmod forgets what interface it should be replying on and ignores the NAT Reflection rules. If I disconnect the Android client and reconnect, everything goes back to normal and it no longer tries to send traffic out the wrong interface.
This exclusively happens on wireguard-kmod because I've have absolutely no issues with wireguard-go. I also don't believe this is a 23.1-specific issue because I experienced the same thing on 22.7 a few months back when I tried to switch to wireguard-kmod, but ultimately had to revert back to wireguard-go.
Hopefully this is enough detail for a developer to reproduce my issue. If you have any questions or need further clarification, please let me know.
-
Anybody else having problems with Wireguard Kernel vs go?
Yes. Upgraded from 22.7 with WG being installed and used with my mobile phone as only client (so far). No issues on 22.7. After the upgrade I found out that any network access of my mobile phone is blocked/stopped if I don't use it for a while (around 15 mins or longer) while the WG client is active and I am connected to my home wifi network. I can't ping anything, my phone doesn't react to ping on the ip address assigned for WG but reacts to ping on the address used while connected to wlan without WG turned on. On top, the GUI of opnsense shows handshakes between server and client all the time long until I start using the phone after a break. Then the handshakes also stop.
I can resolve this by turning the WG client on my phone off and on again. Then my phone has a connection like before until I make another break.
I have switched back to the old module now and haven't run into any issues so far.
-
Isn't this what the "Persistent Keepalive" setting is for?
Sends a heartbeat every X seconds to keep the tunnel up.