FRR Not Redistributing Kernel Routes to OSPF Post Upgrade

[TotalGriffLock]

Hi!

I have 6 OPNsense firewalls which all share IPv4 routes via OSPF. I am part way through upgrading them all from 23.1.11_1 to 23.7.2. No configuration changes have taken place other than upgrading, however I am aware this upgrade moves to FRR 8. What I am seeing is that OSPF on the upgraded firewalls does not redistribute kernel, or static routes despite having the options set in the configuration.

e.g. from inside vtysh

Code: [Select]

fw-1# sh run
Building configuration...

Current configuration:
!
frr version 8.5.2
frr defaults traditional
hostname fw-1
log syslog notifications
!
interface openvpn
 ip ospf passive
exit
!
interface vmx0
 ip ospf passive
exit
!
interface vmx2
 ip ospf cost 200
exit
!
interface vmx3
 ip ospf passive
exit
!
interface vmx4
 ip ospf passive
exit
!
interface vmx5
 ip ospf passive
exit
!
router ospf
 ospf router-id 7.0.1.1
 redistribute kernel
 redistribute connected
 redistribute static
 network 1.2.3.4/16 area 1.1.1.1
 network 10.1.1.0/30 area 1.1.1.1
 network 10.2.2.0/30 area 1.1.1.1
exit
!
end

My kernel routing table:

Code: [Select]

root@fw-1:/var/log # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            x.x.x.x        UG1        vmx1
10.0.0.0/16        127.0.0.1          USB         lo0
10.1.0.0/24        127.0.0.1          USB         lo0
-- snip --
10.2.2.0/24      127.0.0.1          USB         lo0
10.2.3.0/24      127.0.0.1          USB         lo0

My routing table in FRR - note there are only OSPF and Connected routes in the table. Also note I have had to substitute the IP addresses but they are not relevant to the problem:

Code: [Select]

fw-1# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

O>* 0.0.0.0/0 [110/1] via 1.1.1.1, vmx1, weight 1, 00:23:35
O>* 1.1.1.1/28 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:35
O   1.1.1.1/24 [110/10] is directly connected, vmx0, weight 1, 00:23:51
C>* 1.1.1.1/24 [0/1] is directly connected, vmx0, 00:23:51
O   1.1.1.1/24 [110/10] is directly connected, vmx3, weight 1, 00:23:51
C>* 1.1.1.1/24 [0/1] is directly connected, vmx3, 00:23:51
O   1.1.1.1/24 [110/10] is directly connected, vmx4, weight 1, 00:23:51
C>* 1.1.1.1/24 [0/1] is directly connected, vmx4, 00:23:51
O>* 1.1.1.1/28 [110/120] via 1.1.1.1, vmx1, weight 1, 00:23:36
O   1.1.1.1/26 [110/10] is directly connected, ovpns1, weight 1, 00:23:51
C>* 1.1.1.1/26 [0/1] is directly connected, ovpns1, 00:23:51
O   1.1.1.1/30 [110/10] is directly connected, vmx5, weight 1, 00:23:51
C>* 1.1.1.1/30 [0/1] is directly connected, vmx5, 00:23:51
O>* 1.1.1.1/30 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:35
O   1.1.1.1/30 [110/10] is directly connected, vmx1, weight 1, 00:23:51
C>* 1.1.1.1/30 [0/1] is directly connected, vmx1, 00:23:51
O>* 1.1.1.1/30 [110/210] via 1.1.1.1, vmx1, weight 1, 00:23:36
O   1.1.1.1/30 [110/200] is directly connected, vmx2, weight 1, 00:23:51
C>* 1.1.1.1/30 [0/1] is directly connected, vmx2, 00:23:51
O>* 1.1.1.1/30 [110/400] via1.1.1.1, vmx2, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/210] via 1.1.1.1, vmx1, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/210] via 1.1.1.1, vmx2, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/400] via 1.1.1.1, vmx2, weight 1, 00:23:36
O>* 1.1.1.1/32 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:35

Yet here is the routing table from within FRR/vtysh on a firewall which has NOT been upgraded, running the same configuration. As it has not been upgraded it runs FRR 7.5.1

Code: [Select]

fw-2# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

O>* 0.0.0.0/0 [110/1] via 1.1.1.1, vmx1, weight 1, 00:23:35
O>* 1.1.1.1/28 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:35
O   1.1.1.1/24 [110/10] is directly connected, vmx0, weight 1, 00:23:51
C>* 1.1.1.1/24 [0/1] is directly connected, vmx0, 00:23:51
O   1.1.1.1/24 [110/10] is directly connected, vmx3, weight 1, 00:23:51
C>* 1.1.1.1/24 [0/1] is directly connected, vmx3, 00:23:51
O   1.1.1.1/24 [110/10] is directly connected, vmx4, weight 1, 00:23:51
C>* 1.1.1.1/24 [0/1] is directly connected, vmx4, 00:23:51
O>* 1.1.1.1/28 [110/120] via 1.1.1.1, vmx1, weight 1, 00:23:36
O   1.1.1.1/26 [110/10] is directly connected, ovpns1, weight 1, 00:23:51
C>* 1.1.1.1/26 [0/1] is directly connected, ovpns1, 00:23:51
O   1.1.1.1/30 [110/10] is directly connected, vmx5, weight 1, 00:23:51
C>* 1.1.1.1/30 [0/1] is directly connected, vmx5, 00:23:51
O>* 1.1.1.1/30 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:35
O   1.1.1.1/30 [110/10] is directly connected, vmx1, weight 1, 00:23:51
C>* 1.1.1.1/30 [0/1] is directly connected, vmx1, 00:23:51
O>* 1.1.1.1/30 [110/210] via 1.1.1.1, vmx1, weight 1, 00:23:36
O   1.1.1.1/30 [110/200] is directly connected, vmx2, weight 1, 00:23:51
C>* 1.1.1.1/30 [0/1] is directly connected, vmx2, 00:23:51
O>* 1.1.1.1/30 [110/400] via1.1.1.1, vmx2, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/210] via 1.1.1.1, vmx1, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/210] via 1.1.1.1, vmx2, weight 1, 00:23:36
O>* 1.1.1.1/30 [110/400] via 1.1.1.1, vmx2, weight 1, 00:23:36
O>* 1.1.1.1/32 [110/20] via 1.1.1.1, vmx1, weight 1, 00:23:35
K>* 10.0.0.0/16 [0/0] unreachable (blackhole), 01:32:28
K>* 10.1.0.0/24 [0/0] unreachable (blackhole), 01:32:28
K>* 10.2.2.0/24 [0/0] unreachable (blackhole), 01:32:28
K>* 10.2.3.0/24 [0/0] unreachable (blackhole), 01:32:28

Same config file, different version of FRR, different result. Does FRR 8 no longer redistribute blackhole/unreachable routes? I use this to inject routes to policy-based VPN tunnels into the OSPF process.

[newsense]

Check in OSPF - General Tab - Route Redistribution - and enable Kernel Routes in the drop down menu. Seems to be Off by default

[TotalGriffLock]

Thanks for the suggestion, but this setting is still on post upgrade - it's what puts the redistribute lines into the ospfd.conf file. Configuration hasn't changed, not in opnsense or in frr.

[newsense]

Looks like there's an incomplete issue raised already on Github, might be best to comment there as well or open a new one referencing it

https://github.com/opnsense/plugins/issues/3543

[mimugmail]

As only frr changed from 7 to 8 maybe someone can search the web if there changed anything inside frr or needs some additional configs