OPNsense Forum

Archive => 22.1 Legacy Series => Topic started by: jchnnz on March 25, 2022, 08:01:46 am

Title: OSPFv3 fails in 22.1.3 - seems to be PPPoE related?
Post by: jchnnz on March 25, 2022, 08:01:46 am
I'm coming back to OPNsense after switching to pfSense for a couple of years due not being able to get more than ~1.2Gbps throughput (whereas with pfSense I'm able to get ~4Gbps). Looks like 22.1/FreeBSD 13 has fixed the root cause as in my testing I'm now able to push ~6Gbps, which is awesome. Platform is ESXi 7.0U3 with VMXNET3 interfaces.

There's only one thing holding up the cutover - OSPFv3 seems to break (and all IPv6 routes end up expiring) when I shutdown pfSense and bring up the PPPoE connection on OPNsense. It really has me stumped.

Of note, OSPFv3 is working fine with pfSense and OSPF (i.e. for IPv4) works fine across both pfSense and OPNsense. The version of FRR is also the same across both platforms (7.5.1_3). Socket buffers are also set to 16777216. The last version of OPNsense I used was 20.1 (IIRC) and OSPFv3 was working fine back then.

For the cutover I'm doing the following (and there's a diagram of the network layout that hopefully helps):

The logs don't show anything useful, even at debug level. I've performed the following troubleshooting

I believe it's related to PPPoE for the following two reasons:

Config is below:
Code: [Select]
Current configuration:
!
frr version 7.5.1
frr defaults traditional
hostname core-fw1
log syslog
!
interface vmx1
 ip ospf area 0.0.0.0
 ip ospf priority 200
 ipv6 ospf6 network broadcast
 ipv6 ospf6 priority 200
!
router ospf
 ospf router-id 192.168.2.254
 redistribute kernel
 redistribute connected
 redistribute static
 default-information originate always
!
router ospf6
 ospf6 router-id 192.168.2.254
 redistribute kernel
 redistribute connected
 redistribute static
 interface vmx1 area 0.0.0.0
!
line vty
!
end

Is there something I've missed? Something else I should try? Or is this a bug?
Title: Re: OSPFv3 fails in 22.1.3 - seems to be PPPoE related?
Post by: mimugmail on March 25, 2022, 09:14:13 am
PLease remove area config in interface section and add areas to network tab.
Title: Re: OSPFv3 fails in 22.1.3 - seems to be PPPoE related?
Post by: jchnnz on March 25, 2022, 09:41:03 am
There's no Networks tab for OSPFv3. Only Interfaces.
Title: Re: OSPFv3 fails in 22.1.3 - seems to be PPPoE related?
Post by: jchnnz on March 25, 2022, 03:06:50 pm
I've done some further testing...

- I replicated the config from pfSense through vtysh rather than the GUI. Not all commands stuck in the running config, possibly because they're default values. The config seems to survive a restart of routing, but doesn't seem to survive a reboot.

- If I leave pppoe0 disconnected, I can restart routing through the GUI as often as I want and both OSPF (IPv4) and OSPFv3 (IPv6) work as expected. Within 15 or so seconds all the neighbours communicate and start sharing routes. Internal IPv6 endpoints become reachable. Can't quite say indefinitely, but everything kept working for 20mins which seems a sufficiently long test

- If I then establish a pppoe session, both OSPF (IPv4) and OSPFv3 (IPv6) continue to work. However, after approx 10 mins IPv6 routes drop out of the routing tables of OPNsense and the downstream devices. OSPFv3 becomes dead in the water

- Simply disconnecting the pppoe session and restarting routing has no effect. To restore OSPFv3 I need remove the OSPFv3 config (no router ospf6 in vtysh), write out the config, restart routing, recreate the config, write out the config, and then restart routing.

- If routing is restarted from the GUI while a pppoe session is active OSPFv3 will never recover

- If OPNsense is rebooted and prevented from establishing a pppoe session, OSPFv3 starts up and operates normally

- If OPNsense is rebooted normally, OSPFv3 never start operating properly

There seems to be some sort of bug or conflict between pppoe and ospf6d...