Wireguard kernel not working like it should

[RamSense]

@tiermutter thnx for testing! I have not timed it exactly, but it was after several hours. If i head to guess I cap it at 4 hours. Would be great to hear back if it gets broken at your end also, or keeps on working.

[tiermutter]

Ok, I'll keep on testing. WG is now up since about 02:30 pm.

Do you connect to WG server with ipv4 or v6? I am on v6.

[tiermutter]

mhhh... looks like there was an issue shortly before I sent my last post, but it seems to be "my" fault:
For a short time I stayed on a place where wifi signal is not very good and WG stopped saying

Code Select Expand

Failed to send data packet: write udp6 [::]:53889->[2a00:xxxx:xxxx:xxxx::xxxx]:55190: sendto: network is unreachable  ::)

Now tunnel is up since 04:28 pm..... now I also activated the tunnel on my PC.

[RamSense]

thanks for the update. I have my clients config set with both ipv6 and ipv4 addresses. I think that the ipv6 get's picked by preference or because it is stated as first and ipv4 as second address.
To make it even more weird, at the time of error, some iphone's with wg and wifi where still working, and others were not, and after the opnsense reboot, and second failure moment another device what was working the first time, was now also failing.
where do you find the log info

Failed to send data packet: write udp6 [::]:53889->[2a00:xxxx:xxxx:xxxx::xxxx]:55190: sendto: network is unreachable

?

[tiermutter]

I have my clients config set with both ipv6 and ipv4 addresses.

Yes, v4 and v6 inside the tunnel... but how do you estabish a connection to the WG server?

where do you find the log info

It's on Android/ Settings (three dots upper right).

[RamSense]

but how do you estabish a connection to the WG server

by my own domainname referring back to opnsense with it's ipv6 and ipv4 ip.

It's on Android/ Settings (three dots upper right)

ah ok, i'm on iphone and the wireguard app log is not going back long enough to go back tot the error moment. I will keep that in mind when i'll go for a retest at some point.

[tiermutter]

by my own domainname referring back to opnsense with it's ipv6 and ipv4 ip.

Ah ok, sorry, now I understand :)

[tiermutter]

Six hours later nothing happend, everything is working fine, Windows and Android client are still connected via IPv6 GUA. Watching the WG server status I remember that my NAS is always connected as client via IPv6 ULA. Since update / reboot it's about 8:45 hours.

[RamSense]

thanks for the update and reporting back! I have just installed the kernel version again. After installation I noticed that the service widget was still working, while with the initial update of opnsense it was not / showing red. While the wireguard go version gets removed, I may assume that directly after the install of the kernel version all connected vpn devices are connected by the kernel version and not the removed go version.
Nevertheless, I installed the opnsense-patch -c plugins 2ed1f987eb97d right after installing wireguard kernel.

I will test it now and see what happens. When something happens I will try to find something in the log.
Will report back

p.s. have you also tested if wireguard kernel keeps on working after rebooting the opnsense box?

[tiermutter]

The patch makes the service widget show the correct status of WG kmod, that should have nothing to do with your problems I guess.

No, havn't rebooted yet, maybe late this evening when I am calibrating my UPS after battery change...

[RamSense]

The patch makes the service widget show the correct status of WG kmod, that should have nothing to do with your problems I guess.

Exactly, that is what i meant. The service widget was already showing the correct status after installing wireguard kernal (what deinstalled the go version automatically), and before i installed the patch. That was the first different behavior to the upgrade of opnsense. With the opnsense update and installed wireguard kernel, the service widget was not working / showing wireguard as down, while the kernel version was running.

I'm very curious what happens after your reboot, read in another thread of the wg interface being reported down (wg0) at boot https://forum.opnsense.org/index.php?topic=31889.msg155319#msg155319

[RamSense]

It has been almost 4 hours and still all systems go.... Makes me wonder about 2 things:

1. Does your wireguard kernel keeps working after a reboot?
2. Can it be that the opnsense update did something different than the manual installation of wireguard kernel (with auto removing wireguard go) ?

[tiermutter]

Did a reboot and again everything is working fine  :)
Don't know if there is a diference between new install and update...

[Boil2310]

I'm experiencing a similar issue after upgrading to 23.1 where Wireguard handshakes are timing out when at home and decided to do some debugging.

My Android Wireguard client is setup pointing at a hostname, vpn.mydomain.com:51820, which is an A record pointing at my public IP, and I use the Always-on VPN feature in Android on this tunnel. I have all 3 of the NAT Reflection settings in OPNsense's settings (under Firewall > Settings > Advanced) turned on.

igb1 is my LAN, igb2 is my WAN, and wg1 is the Wireguard interface. When I caught the Android client sending handshakes and timing out, I turned on debugging for wg1 (ifconfig wg1 debug) which showed that OPNsense was receiving the handshake and sending a reply to the client, which led to me dig deeper.


wg1: Receiving handshake initiation from peer 1
wg1: Sending handshake response to peer 1
wg1: Receiving handshake initiation from peer 1
wg1: Sending handshake response to peer 1


I checked tcpdump on igb1 and I was able to see the handshake packets from my phone (192.168.1.68) directed to my public IP (let's call it 203.0.113.7), however there was no traffic flowing back to the phone:


root@opnsense:~ # tcpdump -nn -i igb1 host 192.168.1.68 and port 51820
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb1, link-type EN10MB (Ethernet), capture size 262144 bytes
10:45:49.935640 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
10:45:54.967559 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
10:46:03.355900 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148
10:46:11.883729 IP 192.168.1.68.35190 > 203.0.113.7.51820: UDP, length 148


I then checked igb2 and noticed that it is sending the traffic destined for my LAN out the WAN interface:


root@opnsense:~ # tcpdump -nn -i igb2 host 192.168.1.68 and port 51820
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb2, link-type EN10MB (Ethernet), capture size 262144 bytes
11:14:38.316335 IP 203.0.113.7.51820 > 192.168.1.68.35190: UDP, length 92
11:14:43.354698 IP 203.0.113.7.51820 > 192.168.1.68.35190: UDP, length 92


It seems that after some indeterminate period of time, wireguard-kmod forgets what interface it should be replying on and ignores the NAT Reflection rules. If I disconnect the Android client and reconnect, everything goes back to normal and it no longer tries to send traffic out the wrong interface.

This exclusively happens on wireguard-kmod because I've have absolutely no issues with wireguard-go. I also don't believe this is a 23.1-specific issue because I experienced the same thing on 22.7 a few months back when I tried to switch to wireguard-kmod, but ultimately had to revert back to wireguard-go.

Hopefully this is enough detail for a developer to reproduce my issue. If you have any questions or need further clarification, please let me know.

[becks0815]

Anybody else having problems with Wireguard Kernel vs go?

Yes. Upgraded from 22.7 with WG being installed and used with my mobile phone as only client (so far). No issues on 22.7. After the upgrade I found out that any network access of my mobile phone is blocked/stopped if I don't use it for a while (around 15 mins or longer) while the WG client is active and I am connected to my home wifi network. I can't ping anything, my phone doesn't react to ping on the ip address assigned for WG but reacts to ping on the address used while connected to wlan without WG turned on. On top, the GUI of opnsense shows handshakes between server and client all the time long until I start using the phone after a break. Then the handshakes also stop.

I can resolve this by turning the WG client on my phone off and on again. Then my phone has a connection like before until I make another break.

I have switched back to the old module now and haven't run into any issues so far.