[FIXED] Wireguard multi site to site with FRR routing allowed ips: (none)

[nzkiwi68]

I have 3 sites, each site has multi WAN.

Site A: WAN1 & WAN2 plus HA firewall pair
Site B: WAN1 & WAN2 plus HA firewall pair
Site C: WAN1 & WAN2 plus HA firewall pair

I'm using FRR with BGP for dynamic routing and got it working great with 2 sites and excellent WAN failover, only losing 2 pings during WAN failover. As soon as I added a third site, I get a strange "allowed ips: (none)" and routing problems.

The allowed ips set for the peer is 0.0.0.0/0, but, it seems wireguard doesn't tolerate more peers with allowed ips of 0.0.0.0/0 set against the same local listener.

The reason I want to set 0.0.0.0/0 is I want to do all my routing using FRR, so, I don't want to have to set the peer allowed IP addresses in wireguard plus then control the IP addresses in FRR BGP.

See the attachment, it shows that the running config for the peer smPI... has allowed ips: (none) (but I can assure you, it has allowed ips of 0.0.0.0/0 set) and routing doesn't work. As soon as I put a list of allowed ips as expected from that peer, volia, it works.

The local endpoint (listener) of course has disable routes set.

FRR and BGP and BFD all working great.

Environment

• OPNsense 22.7.2
• Wireguard
• Wireguard-kmod
• 10-wireguard CARP hook script

[nzkiwi68]

Any comments?

Can I provide any further information or clarification?

[Maurice]

it seems wireguard doesn't tolerate more peers with allowed ips of 0.0.0.0/0 set against the same local listener.

Correct. You can't assign multiple endpoints with overlapping allowed IPs to the same local wg interface. That's just not how WireGuard works. To put it simply, it uses the IP address to determine which key to use for encryption / decryption.

For multiple 0.0.0.0/0 endpoints, you need to create an individual local wg interface for each endpoint.

Cheers
Maurice

[nzkiwi68]

 

Thanks!

I get it now, I have created multiple WG local listeners, one for each endpoint.
In the end, I have not gone with the 0.0.0.0/0 but have put the specific expected remote networks and listed these against each of the wireguard peers.

I have done that because once one of the peers became a preferred gateway for traffic that should not have been sent there, probably a weird combination of starting and stopping FRR and incorrectly me setting the gateway priority lower on a gateway to the remote FRR BGP endpoint.

Now using 0.0.0.0/0 for the remote peer is a bit more work for endpoint config to specify the allowed networks but safer that the local FRR will not inject perhaps a route all to remote WG peer that you don't want.

I now have 3 sites:

• All OPNsense HA
• With dual WAN at each site
• WG tunnels for site to site and multi WAN
• FRR with BGP and BFD for routing
• Wireguard-kmod plus the 10-wireguard CARP hook script

It works!

• Pull any WAN on any site, you lose max 2 pings before recovery
• Shutdown any site primary firewall. You lose approx 2 pings before recovery
• Restart primary firewall, fail back is very seamless, only 1-2 ping lost

Very happy. Thanks for your pointer.