[SOLVED] OSPF over WireGuard links has stopped working

[astuckey]

[SOLVED]
https://github.com/opnsense/plugins/issues/2314#issuecomment-851009622
Upgrade to 21.1.6 fixed the issue.


Hi there,

I have been using OSPF over WG for over a year, but it seems to have broken in one of the past few releases, 21.1.3 I believe I started seeing my issue.

I'm currently 21.1.5 in a VM.

It appears the initial messaging for OSPF is having the problem, as no neighbours are discovered, and some very ugly messages in the logs:

May 20 15:57:37 <host-removed> frr_carp[90878]: FRR received carp configuration event.
May 20 15:57:37 <host-removed> ospfd[21850]: [EC 100663299] setsockopt_so_sendbuf: fd 8: SO_SNDBUF set to 2097152 (requested 8388608)
May 20 15:57:37 <host-removed> ospfd[21850]: [EC 100663299] setsockopt_so_recvbuf: fd 8: SO_RCVBUF set to 2097152 (requested 8388608)
May 20 15:57:37 <host-removed> ospfd[21850]: ASBR[default:Status:1]: Update
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] setsockopt_so_sendbuf: fd 13: SO_SNDBUF set to 2097152 (requested 8388608)
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] setsockopt_so_recvbuf: fd 13: SO_RCVBUF set to 2097152 (requested 8388608)
May 20 15:57:37 <host-removed> ospfd[2077]: ASBR[default:Status:1]: Update
May 20 15:57:37 <host-removed> zebra[83367]: client 19 says hello and bids fair to announce only ospf routes vrf=0
May 20 15:57:37 <host-removed> frr_carp[19057]: FRR received carp configuration event.
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_ADD_MEMBERSHIP (fd 13, addr 169.254.0.1, ifindex 7, AllSPFRouters): Can't assign requested address; perhaps a kernel limit on # of multicast group memberships has been exceeded?
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_ADD_MEMBERSHIP (fd 13, addr 169.254.192.2, ifindex 8, AllSPFRouters): Can't assign requested address; perhaps a kernel limit on # of multicast group memberships has been exceeded?
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_MULTICAST_IF(fd 13, addr 169.254.0.1, ifindex 7): Operation not supported
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_MULTICAST_IF(fd 13, addr 169.254.192.2, ifindex : Operation not supported
May 20 15:57:37 <host-removed> ospfd[2077]: [EC 100663299] *** sendmsg in ospf_write failed to 224.0.0.5, id 0, off 0, len 64, interface wg1, mtu 1420: Network is unreachable
May 20 15:57:37 <host-removed> ospfd[2077]: LSA[Type5:0.0.0.0]: Not originate AS-external-LSA for default
May 20 15:57:38 <host-removed> frr_carp[19057]: FRR trigger OspfdEventHandler event.
May 20 15:57:47 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_MULTICAST_IF(fd 13, addr 169.254.0.1, ifindex 7): Operation not supported
May 20 15:57:47 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_MULTICAST_IF(fd 13, addr 169.254.192.2, ifindex : Operation not supported
May 20 15:57:47 <host-removed> ospfd[2077]: [EC 100663299] *** sendmsg in ospf_write failed to 224.0.0.5, id 0, off 0, len 64, interface wg1, mtu 1420: Network is unreachable
May 20 15:57:57 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_MULTICAST_IF(fd 13, addr 169.254.0.1, ifindex 7): Operation not supported
May 20 15:57:57 <host-removed> ospfd[2077]: [EC 100663299] can't setsockopt IP_MULTICAST_IF(fd 13, addr 169.254.192.2, ifindex : Operation not supported
May 20 15:57:57 <host-removed> ospfd[2077]: [EC 100663299] *** sendmsg in ospf_write failed to 224.0.0.5, id 0, off 0, len 64, interface wg1, mtu 1420: Network is unreachable

The WG links are up, set up as gateways with monitored pings.  Static routing works fine over the links, just OSPF seems to now struggle.

Configuration of one side:

Building configuration...

Current configuration:
!
frr version 7.4
frr defaults traditional
hostname <host-removed>
log syslog informational
!
interface wg0
 ip ospf area 0.0.0.0
 ip ospf network point-to-point
!
interface wg1
 ip ospf area 0.0.0.0
 ip ospf network point-to-point
!
router ospf
 ospf router-id <ip-removed>
 redistribute kernel
 passive-interface lo0
 passive-interface vtnet0
 passive-interface vtnet1
 passive-interface wg5
 passive-interface wg7
 passive-interface wg8
 passive-interface wireguard
!
line vty
!
end

And a second node (3 nodes in a ring):

Building configuration...

Current configuration:
!
frr version 7.4
frr defaults traditional
hostname <host-removed>
log syslog informational
!
interface wg0
 ip ospf area 0.0.0.0
 ip ospf network point-to-point
!
interface wg1
 ip ospf area 0.0.0.0
 ip ospf network point-to-point
!
router ospf
 ospf router-id <ip-removed>
 redistribute kernel
 passive-interface lo0
 passive-interface vtnet0
 passive-interface vtnet1
 passive-interface wg5
 passive-interface wg7
 passive-interface wireguard
!
line vty
!
end

Just wondering if anyone has any tips on troubleshooting this further?

Thanks,
Adam

[astuckey]

The WG config:

Node A:

[Interface]
PrivateKey = <removed>
Address = 169.254.0.1/30
ListenPort = 51820
Table = off
PostUp = route add 169.254.0.2 -iface %i
PostDown = route del 169.254.0.2 -iface %i

[Peer]
PublicKey = <removed>



PresharedKey = <removed>
Endpoint = <removed>
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 120

Note B:

[Interface]
PrivateKey = <removed>
Address = 169.254.0.2/30
ListenPort = 51821
Table = off
PostUp = route add 169.254.0.1 -iface %i
PostDown = route del 169.254.0.1 -iface %i

[Peer]
PublicKey = <removed>



PresharedKey = <removed>
Endpoint = <removed>
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 120

Firewall rule for the "WireGuard" generic interface is to allow all IPV4 for in/out.

[astuckey]

Looks like FRR was broken.  An update was just released, upgrading to 21.1.6 has fixed the issue.

https://github.com/opnsense/plugins/issues/2314#issuecomment-851009622