Hello everyone,
I've been experimenting with OPNsense for a few days to create a hub that securely connects to remote sites using both WireGuard and ZeroTier, while distributing routes via OSPF.
In addition to sites equipped with advanced routing systems, I also have three sites with basic routers that support WireGuard tunnels. To address this, I set up a WireGuard instance in the hub with three peers for these sites. After verifying connectivity through the tunnels, my plan was to distribute the routes for these sites via OSPF.
Unfortunately, despite trying several options for enabling static and/or kernel routes, these routes are not being advertised. Specifically:
1)
With "Disable routes" unchecked in the WireGuard instance:
- Routes for the site networks are visible using netstat -rn.
- Routing from OPNsense to the sites works.
- However, vtysh doesn't display any kernel/static route apart from some Cloudflare DNS server entries (likely derived from the WAN interface).
Destination Gateway Flags Netif Expire
0.0.0.0/1 link#8 US wg0
default 10.a.b.1 UGS hn0
1.0.0.2 10.a.b.1 UGHS hn0
1.1.1.2 10.c.d.1 UGHS hn0
10.e.f.1 link#8 UHS wg0
10.g.h.0/24 link#8 US wg0
10.i.j.0/24 link#8 US wg0
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
K>* 0.0.0.0/0 [0/0] via 10.a.b.1, hn0, 06:53:28
K>* 1.0.0.2/32 [0/0] via 10.a.b.1, hn0, 06:53:28
K>* 1.1.1.2/32 [0/0] via 10.a.b.1, hn0, 06:53:28
wg0: flags=10080c1<UP,RUNNING,NOARP,MULTICAST,LOWER_UP> metric 0 mtu 1420
description: bk_wg (opt2)
options=80000<LINKSTATE>
inet 10.k.l.129 netmask 0xfffffff0
groups: wg wireguard bk
nd6 options=1<PERFORMNUD>
2)
With a gateway set up in System/Gateways and a static route in System/Routes:
- A kernel route appears in vtysh but is marked "inactive."
- There's no change in netstat -rn.
K 10.g.h.0/24 [0/0] via 10.g.h.1 inactive, 00:00:49
3)
With a static route added in Routing/OSPF/Static:
- A static route appears in vtysh but is still marked "inactive."
S 10.g.h.0/24 [1/0] via 10.g.h.1, wg0 inactive, weight 1, 00:00:37
In all cases, routes fail to appear in OSPF. I've tried multiple configurations in OSPF without filters (filters could be a mystery for another night). I suspect this behavior is related to these routes being considered "inactive," preventing them from being advertised by OSPF.
Here is the current running configuration for FRR:
!
frr version 8.5.7
frr defaults traditional
hostname gw.lan....
log syslog notifications
!
ip route 10.g.h.0/24 10.g.h.1 wg0
!
interface hn0
ip ospf passive
exit
!
interface hn1
ip ospf passive
exit
!
interface wg1 #### P2P for site with OSPF router
ip ospf area 10.x.0.0
ip ospf network point-to-point
exit
!
interface wg2 #### P2P for site with OSPF router
ip ospf area 10.x.0.0
ip ospf network point-to-point
exit
!
interface ztdmp..... #### ZeroTier L2 Domain for sites with OSPF router
ip ospf area 10.x.0.0
ip ospf network broadcast
exit
!
router ospf
ospf router-id 10.x.0.1
redistribute kernel
redistribute connected
redistribute static
exit
!
bfd
exit
!
end
Thanks for any input!
I think I've figured it out.The routes for the remote networks were "inactive" because they referenced the remote gateway IP address alongside its corresponding remote network IP address.
In other words, in my setup, there isn't a common transit network between the hub and the three less advanced remote gateways, simply because they don't support multiple networks or IP addresses.As a result, the remote gateway was formally unreachable since the remote site network wasn't covered by any other route. While the remote network and gateway were technically reachable via the auto-created routes by WireGuard, it seems these routes couldn't be taken into account.
I'm not certain if there's a concept akin to route scope in OPNsense/FreeBSD, but it appears that WireGuard routes cannot be used for next-hop searching in static routes configured via the System or Routing menu.There are two potential solutions:
- Assign an additional IP address from the remote network to the WireGuard interface. This way, the remote network becomes connected, reachable, and advertised over OSPF without the need to propagate static routes.
- Configure static routes for the remote networks directly in the Routing/Static menu, without specifying the IP address of the remote gateway at the far end of the WireGuard tunnel—using only the WireGuard interface instead. Then, advertise static routes over OSPF as well.
Both approaches require maintaining the remote network IP addresses in two places.
A sample working configuration follows.
!
frr version 8.5.7
frr defaults traditional
hostname gw.lan.mydomain.tld
log syslog notifications
!
ip route 10.a.b.0/24 wg0
ip route 10.a.c.0/24 wg0
ip route 10.d.e.0/24 wg0
!
interface hn0
ip ospf passive
exit
!
interface hn1
ip ospf passive
exit
!
interface wg1
ip ospf area 10.2x.0.0
ip ospf network point-to-point
exit
!
interface wg2
ip ospf area 10.x.0.0
ip ospf network point-to-point
exit
!
interface ztdmpyyyyyyyyyyyyyy
ip ospf area 10.x.0.0
ip ospf network broadcast
exit
!
router ospf
ospf router-id 10.x.0.1
redistribute connected
redistribute static
exit
!
end