Hi guys,
Ive been trying to setup for a week or so wireguard site to site VPN without success. keep getting Handshake errors as bellow, tunnel comes up and peer can be seen but not pingable and no routing is possible
Handshake did not complete after 5 seconds, retrying (try 2)
Sending handshake initiation
Handshake did not complete after 5 seconds, retrying (try 2)
This is between 2 OPNsense boxes, second box, the client has no public access from the outside however it has full outbound internet traffic allowed.
Site A (Main Server) - Has public IP with WAN rule allowing port 51820
[Interface]
Address = 192.168.1.1/24
MTU = 1500
ListenPort = 51820
PrivateKey = XXXXXXXX/7pPnNLvm8I1evXgCoU2z733tzgxL+qve9GM=
[Peer]
PublicKey = XXXXXXXXaJmnOotn3NW1LIYOe60aqqKByp7oEfhltFc=
AllowedIPs = 192.168.1.2/32,10.0.40.0/24
PersistentKeepalive = 20
Site B (full open outbound internet only, no NAT or FW access)
[Interface]
Address = 192.168.1.2/24
MTU = 1500
ListenPort = 27836
PrivateKey = XXXXXXXX1UMOhNzm7cUQamH7MwHBNLs4Ot41mIQ1wlI=
[Peer]
PublicKey = XXXXXXXXuXrQftcGxJzd6DYLW+ovR2HoRnhg1ojykSo=
AllowedIPs = 192.168.1.1/32,172.16.69.0/24
Endpoint = 76.XX.XX.257:51820 (Site A IP and Port)
PersistentKeepalive = 20
List config
interface: wg0
public key: XXXXXXXXuXrQftcGxJzd6DYLW+ovR2HoRnhg1ojykSo=
private key: (hidden)
listening port: 51820
peer: XXXXXXXXaJmnOotn3NW1LIYOe60aqqKByp7oEfhltFc=
preshared key: (hidden)
endpoint: 81.3.249.54:27836
allowed ips: 10.0.40.0/24,192.168.1.2/32
transfer: 46.68 KiB received, 42.32 KiB sent
persistent keepalive: every 20 seconds
wg0 XXXXXaJmnOotn3NW1LIYOe60aqqKByp7oEfhltFc= 0
All I am trying to do is to route 172.16.69.0/24 to 10.0.40.0/24 and vice versa, this should be fairly simple.
OpenVPN works perfectly with those networks, however I wanted to take advantage of the wireguard so called "speed".
I have tried to regenerate the keys at both sides 100 times
any thoughts about what is wrong?
I cannot check you IPs (srly 192.168.1.0/24 as the tunnel network?) and certificates, but I would remove the MTU.
Persistent keepalive not needed, as added automagically by OPNsense when configured via GUI. Preshared Key I would remove at this stage.
Have here wireguard up and running between 2x OPNsense. One site needs a firewall rule on WAN (51820 or 27836, chose one) for UDP. Then it should work imho.
PS: If you have the appropriate firewall rules on both Wireguard interfaces.
Yes MTU setting is just out of desperation, this should by all means be the easiest VPN to setup up, hence makes no sense not working or pingign either the VPN peer or any subnet behind the tunnel.
as you can see the tunnel comes up and there is traffic listed but cannot ping anything.
Server side has a rule on WAN to allow UDP on the server port 51820 already, Client outbound is open hence the client connects to server and establishes the tunnel, this seems like a allow list problem or routing problem or even keys.
I don't know enough about wireguard to know where to go next.
Do your LAN rules on both sides allow traffic to the respective remote LANs?
Quote from: chemlud on February 11, 2021, 06:19:52 PM
Do your LAN rules on both sides allow traffic to the respective remote LANs?
Uploaded my complete config in screenshots
Server:
Server
Client:
Quote from: chemlud on February 11, 2021, 06:19:52 PM
Do your LAN rules on both sides allow traffic to the respective remote LANs?
I haven't touch the LAN interface rules, I have several OpenVPN working with no rules on LAN, whenever I try to reach remote OpenVPN subnets from LAN it just works, I understand wireguard works different than OpenVPN so I might need those rules specifically allowing ?
I have a rule that says From LAN goes everywhere so should work by default
Do you see Routes in System: Routes: Status with the networks mentioned?
Quote from: Gauss23 on February 11, 2021, 06:45:53 PM
Do you see Routes in System: Routes: Status with the networks mentioned?
yes on client I see the remote subnet and on server I see the client subnet
Also I have a any to any rule on the wireguard interface on both locations
After turning off openVPN I wold do a reboot of the OPNsense before trying wireguard...
Quote from: Gauss23 on February 11, 2021, 06:45:53 PM
Do you see Routes in System: Routes: Status with the networks mentioned?
I have tried windows client just to try to troubleshoot and when I try to connect it says handshake failed, waiting for retry, so not sure if this is a bug with keys or something else, the tunnel seems up and traffic listed as passing but no routing, unless I failed miserably in one of the steps I should be able to ping remote subnets or at lest the remote peer IP itself
You should see handshakes.
Maybe try a tcpdump and see where the packets are going. Enable logging on the firewall rules that allow WireGuard traffic and have a look at the live view.
Quote from: chemlud on February 11, 2021, 06:49:07 PM
After turning off openVPN I wold do a reboot of the OPNsense before trying wireguard...
I have rebooted both peers 20 times today...
this is my windows output
Handshake errors are not good. You should be able to connect with that client.
WAN has a rule to allow that traffic? Do you see anything in the live view? Enable logging on rules, that have something to do with your traffic.
And the public key has been correctly generated for the respective private key? I have seen yesterday a wireguard with traffic going back and forth but not getting the handshake done before actually sending some traffic from a LAN client.
Quote from: Gauss23 on February 11, 2021, 06:57:06 PM
Handshake errors are not good. You should be able to connect with that client.
WAN has a rule to allow that traffic? Do you see anything in the live view? Enable logging on rules, that have something to do with your traffic.
the WAN rule is working because the tunnel comes up, you can see the rule bellow
from server I try to reach the client peer on 172.20.20.2 and it passes, but doesn't reach the other side, same problem with client trying to reach 172.20.20.1 on server side
Please enable logging for the WAN rule. You should see the traffic arriving.
And just because traffic is leaving through the wg0 interface doesn't mean it's reaching its destination. WG is stateless, unlike OpenVPN.
Quote from: chemlud on February 11, 2021, 06:57:24 PM
And the public key has been correctly generated for the respective private key? I have seen yesterday a wireguard with traffic going back and forth but not getting the handshake done before actually sending some traffic from a LAN client.
ok interesting, Pease explain, I am generating the keys simply by emptying the public and private fields on the local menu on server and client
then copying the pub key and pasting on the peer side at each location, meaning server Pub key goes to the client peer config as the server to connect to and client pub key goes on the peer config of the server on the endpoint menu
Quote from: Gauss23 on February 11, 2021, 07:12:53 PM
Please enable logging for the WAN rule. You should see the traffic arriving.
And just because traffic is leaving through the wg0 interface doesn't mean it's reaching its destination. WG is stateless, unlike OpenVPN.
ok further interesting, what does stateless mean exactly and how it differs form OpenVPN that is running also on the same server fine
OpenVPN can tell if a connection is living or not.
With WireGuard every packet is on its own. You can only tell when the last handshake took place
Quote from: akron on February 11, 2021, 07:13:50 PM
ok interesting, Pease explain, I am generating the keys simply by emptying the public and private fields on the local menu on server and client
then copying the pub key and pasting on the peer side at each location, meaning server Pub key goes to the client peer config as the server to connect to and client pub key goes on the peer config of the server on the endpoint menu
I always generate both key pairs on another linux machine (never take the first one ;-) )
$ (umask 077 && wg genkey > wg-server-private.key)
$ wg pubkey < wg-server-private.key > wg-server-public.keyand
$ (umask 077 && wg genkey > wg-client-private.key)
$ wg pubkey < wg-client-private.key > wg-client-public.key..and don't forget to delete the files after configuring
ok I may be into something, the word "stateless" rings alarm bells on my rule setup as this is a bit messy at present
if the problem is what I think it will be I have no one to blame but myself for overlooking this.
I'm relatively new to WG and always used OpenVPN without issues since forever
will clean up and start fresh
Quote from: Gauss23 on February 11, 2021, 07:26:19 PM
OpenVPN can tell if a connection is living or not.
With WireGuard every packet is on its own. You can only tell when the last handshake took place
ok I had some sort of success, however not sure how to fix it on a permanent basis
the problem was because I use multiple VIPs on the Site A public WAN interface, the VPN public IP is one of those VIPs, if I use the actually IP of the network interface the tunnel comes up and I can ping both locations subnets, however I would prefer to use a dedicated VIP for WG and not the default public IP.
OpenVPN been working in this manned without any problems, although the client connects to a specific public IP, the reply comes on a different IP which is the default IP of the firewall. This behaviour obviously did not work on WG and it wont work if the reply comes on a different IP than the default outbound WAN interface IP.
I guess what I need to do is to create an outbound rule on server side to make sure the replica back to the client goes on the actual VIP public IP and not the default one to fix this problem?
like this: WAN WireGuard net * * 51820 185.xx VIP IP * YES WG
As far as I know that's currently not possible.
Quote from: Gauss23 on February 11, 2021, 09:12:21 PM
As far as I know that's currently not possible.
ok that is interesting, so this is a know limitation of the WG on OPNsense platform or is in all platforms?
I am still confused how WG works only by using the default public IP of the firewall and not a simple VIP part of that same WAN interface?
there isn't a way or outbound rule that can be created to force the reply back to the client to go on the same public IP as the IP client connects to the server?
not a killer problem but would be nice to know if we can escape this situation with a outbound NAT rule on the server side.
still trying to figure it out why this doesn't affect OpenVPN tunnels, how does the client know that the reply comes on a different IP than the one he connects to as the server?
very confusing for me indeed...
Quote from: Gauss23 on February 11, 2021, 09:12:21 PM
As far as I know that's currently not possible.
Just a quick update, thanks for all the help, I got to the bottom of the problem by simply using the default WAN public IP for the tunnel and not one of the several VIPs I use on the WAN interface.
maybe others have the same problem and know how to fix it, however as far I tested the WG site to site tunnel only works when using the default public IP address of the firewall doesn't work if we use public IP VIPs or alias.
if anyone knows a work around would be nice.
Its unsupported by WireGuard, already asked Jason several times
Quote from: mimugmail on February 12, 2021, 09:06:52 PM
Its unsupported by WireGuard, already asked Jason several times
yes I got to the bottom of it all, using only default WAN IP, also have outbound internet over the tunnel working and NAT port forwarding back over the tunnel to clients working
one small tip for other users experiencing problems, you need to setup manual MSS value on destination WG interface otherwise the TCP traffic wont work properly from client behind the tunnel, not sure why but I found this if in one of mimugmail posts about wireguard MTU and MSS issues
all looking good so far, retired all OpenVPN tunnels
Can you post a screenshot of this portforward please?
Quote from: mimugmail on February 23, 2021, 03:10:06 PM
Can you post a screenshot of this portforward please?
this is how I have it:
Assuming you already have Site 1 (Client) LAN + Internet going over Site 2 (Server) Public IP and you want to NAT something from that public IP back to the client LAN
- Port forward rule
- WAN rule automatically created by Port forward rule
- Then Outbound NAT rule on server side to remote LAN Alias/IP on WG interface
not sure if this is the correct way to do NAT over the tunnel but is working sharp
Ah, now I got you. The unsupported thing is when you want to add a port forward on the Tunnel Address to internal