Hi,
i recently set up two opnsense 21.1.3 with wireguard site to site, the tunnel establishment works like a charm. Since i am currently just testing wireguard and already have a connection between those two sites, i use BGP for route exchange over all available paths.
I configured wireguard on both ends with "disable routes" enabled and 0.0.0.0/0 as allowed networks for the endpoint. Wireguard successfully establishes a tunnel, however via this tunnel both sites cannot reach each other (for example via ping) and also cannot establish a routing neighborship via this connection.
I configured both interfaces ip address on opnsense based on the wireguard configuration (in this case 172.31.32.1 & 2 /24).
After studying the routing table, i noticed that the tunnel subnet was not installed at all and the ping to the remote tunnel ip was forwarded via the default route:
root@OPN-01:~ # netstat -rn4
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.10.1 UGS em0
10.54.112.0/24 10.54.112.1 UGS ovpnc1
10.54.112.1 link#12 UH ovpnc1
10.54.112.46 link#12 UHS lo0
172.31.31.2 link#11 UH lo1
172.31.31.2/32 127.0.0.1 UGSB lo0
172.31.32.2 link#13 UH wg0
root@OPN-01:~ # ping 172.31.32.1
PING 172.31.32.1 (172.31.32.1): 56 data bytes
92 bytes from 192.168.10.1: Redirect Host(New addr: 192.168.10.254)
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 91d7 0 0000 3f 01 52f5 192.168.10.20 172.31.32.1
After adding the route to this tunnel subnet manually on both appliances, i could reach the remote tunnel ip via icmp and the BGP session was established:
root@OPN-01:~ # route add 172.31.32.0/24 -iface wg0
add net 172.31.32.0: gateway wg0
root@OPN-01:~ # ping 172.31.32.1
PING 172.31.32.1 (172.31.32.1): 56 data bytes
64 bytes from 172.31.32.1: icmp_seq=0 ttl=64 time=66.344 ms
64 bytes from 172.31.32.1: icmp_seq=1 ttl=64 time=39.939 ms
64 bytes from 172.31.32.1: icmp_seq=2 ttl=64 time=40.095 ms
^C
--- 172.31.32.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 39.939/48.793/66.344/12.411 ms
root@OPN-01:~ # netstat -rn4
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.10.1 UGS em0
10.54.112.0/24 10.54.112.1 UGS ovpnc1
10.54.112.1 link#12 UH ovpnc1
10.54.112.46 link#12 UHS lo0
10.255.10.0/24 172.31.32.1 UG1 wg0
10.255.11.0/24 172.31.32.1 UG1 wg0
10.255.255.24/30 172.31.32.1 UG1 wg0
10.255.255.26/32 172.31.32.1 UG1 wg0
10.255.255.28/30 172.31.32.1 UG1 wg0
100.64.100.0/30 172.31.32.1 UG1 wg0
100.64.255.0/30 172.31.32.1 UG1 wg0
100.64.255.4/30 172.31.32.1 UG1 wg0
100.64.255.8/30 172.31.32.1 UG1 wg0
100.64.255.12/30 172.31.32.1 UG1 wg0
100.65.100.0/30 link#3 U vmx1
100.65.100.2 link#3 UHS lo0
100.65.200.0/30 link#10 U vmx1_vla
100.65.200.2 link#10 UHS lo0
127.0.0.1 link#5 UH lo0
172.31.31.2 link#11 UH lo1
172.31.31.2/32 127.0.0.1 UGSB lo0
172.31.32.0/24 wg0 US wg0
172.31.32.2 link#13 UH wg0
192.168.1.0/24 172.31.32.1 UG1 wg0
192.168.10.0/24 link#1 U em0
192.168.10.20 link#1 UHS lo0
192.168.11.0/24 172.31.32.1 UG1 wg0
192.168.20.0/24 link#9 U vmx0_vla
192.168.20.253 link#9 UHS lo0
192.168.21.0/24 172.31.32.1 UG1 wg0
192.168.30.0/24 link#8 U vmx0_vla
192.168.30.253 link#8 UHS lo0
192.168.79.0/24 172.31.32.1 UG1 wg0
192.168.80.0/24 172.31.32.1 UG1 wg0
192.168.81.0/27 172.31.32.1 UG1 wg0
192.168.81.0/24 172.31.32.1 UG1 wg0
192.168.90.0/24 172.31.32.1 UG1 wg0
192.168.168.0/24 172.31.32.1 UG1 wg0
192.168.169.0/24 172.31.32.1 UG1 wg0
192.168.170.0/24 172.31.32.1 UG1 wg0
192.168.222.1/32 172.31.32.1 UG1 wg0
192.168.222.128/25 172.31.32.1 UG1 wg0
192.168.255.0/30 172.31.32.1 UG1 wg0
Since i have not found any other topic regarding this problem i am curious if anyone has stumbled upon this problem too or if you are aware of this?
It can only work if you add the bgp peer as a gateway since you disable the adding of kernel routes, but after this it should be easy.
The BGP neighborship is established between the tunnel ips, so no eBGP multihop is used here. In comparison, for other ethernet networks with other BGP peers i do not need any gateway and still the whole network is listed in netstat.
I have defined a gateway with the BGP peer's address and used this in different rules and the interface configuration but have not seen any effect in the netstat -rn4 output so far.
A little hint would be much appreciated ;)
Edit: I figured it out myself, did not display the advanced options in wireguard...