Need help with wireguard setup

[Dark-Sider]

Hey guys,

I'm running a wireguard VPN between two dial-up sites in Germany. One provider in M-Net (local carrier in Munich) and the other provider is Deutsche Glasfaser (a FTTH provider, active in several parts of Germany).

For the ease of this post I'll refer to the two sites as MNET and DG. Both sites have installed identical hardware, opnsense is running in an ESXi 6.7 VM on a qotom headless PC (core i7). I have plenty of experience with that hardware and it gets the job done quite well.

Although both sites connect via FTTH (gpon) mnet requires a pppoe connection which featurese IPv4/IPv6 Dual Stack. The IPv6 prefix (a /56) is received via the IPv4 connectivity through the pppoe connection. The pppoe interface does not receive a public but a linklocal IPv6 address. DG just issues IPv4 and IPv6 addresses via DHCP (no pppoe required). The WAN interface also gets a public /128 IPv6 address. Since DG only offers cg NAT for IPv4 connectivity, I'm using wireguard with IPv6 to connect the two sites.

As the IPv6 subnets assigned by both providers are not static, I use dynv6 to update my IPv6 addresses on both sites. Since MNet does not provide a public v6 address on the pppoe/wan I selected the LAN-IPv6-adresses to be served on dynv6. This is checked, and both boxes are pingable with their respective hostnames from the internet using IPv6.

The wireguard part of the configuration was quite straight forward:
- install wirguard
- configure the local part on both boxes
- configure the remote-endpoint on both boxes with the public key of the other box
- activate the endpoints within the respective local config.

For the allowed IP-addresses I selected the corresponding remote network, and the remote tunnel address with /32

MNET uses 172.20.0.0/16, 172.19.0.0.2/32
DG uses 192.168.0.0/24, 172.19.0.0.1/32
as allowed nets. The allowed net is the remote LAN-Subnet of the other box and the other boxe's tunnell address.

I don't need IPv6 within the tunnel so it's only IPv4 for the tunnel.

For the sake of ease both sites have the firewall settings wg0 to IPv4+IPv6 any to any. (Protocol, sourc, dest, port...)

Both boxes run the latest 20.7 opnSense version.

Now the strange part: The tunnel sometimes works and sometimes doesn't work - although the IPv6 prefixes of both sites have not changed and DNS returns the correct IPv6 address. The DNS record also has only an AAAA record configured, to avoid IPv4 connections.

When I look at the wireguard "List Configuration" output on each site i see sth like this:

Code: [Select]

interface: wg0
  public key: XXX
  private key: (hidden)
  listening port: 51820

peer: XXX
  endpoint: [2a00:6020:1000:xxx]:51820
  allowed ips: 172.19.0.2/32, 172.20.0.0/16
  latest handshake: 19 minutes, 38 seconds ago
  transfer: 22.06 KiB received, 14.01 KiB sent
  persistent keepalive: every 1 minute

The other site displays similiar information. If the connection is working, then the received and sent counters go up as expected and traffic passes through the network. But I very often reach a state where both boxes just send packets and the other box won't receive any, or it receives the packet and even sends out an answer (check with tcpdump) but the sending box won't receive any packets.

I also noticed, that the IPv6 address noted in the "peer" section of the List Configuration view on the MNET box, does not match the configured address in the endpoint dialogue or the the resolved address from hostname. (tried both) but keeps displaying the WAN address of the DG box instead of it's LAN address. So it seems to me that wg tries to origniate traffic from DGs WAN instead of LAN IP. I updated the DYNV6 configuration to match the WAN - but no luck there either.

WG runs on 51820 on both boxes and the ports are opened in the Firewall (wan/pppoe) with "THIS FIREWALL" as target. To allow the ICMP-Echo for both boxes I use the same rule but with ICMP...

any ideas? any fundamental flaws in my thinking?

the odd thing is, that it sometimes works and sometimes doesn't, and I can't see whe

regards,
Fabian

[mimugmail]

Mnet as master and endpoint on mnet firewall without IP, at DG you put mnet IP in endpoint. And set keepalive to 5 seconds, not one minute

[Dark-Sider]

Thanks mate, made the changes and at a first glance it works.

Is this the recommended setup to have one box as "master" and let the "client" (DG) initiate the connection?
https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html -- I assumed that the connection settings have to be mutual. Giving it a second thought though, roadwarriors don't have reachable DNS entries well...

[mimugmail]

It doesn't sound stable when a site with a fixed IPv6 address tries to conntect or only reponds to a CGN DS-lite thingy. So it's better to let the site with the "unreliable" line connect and and the other to respond