[SOLVED] WAN PPPoE IPv6 default route set to wrong interface

[Taomyn]

After many years my ISP has finally sorted IPv6 for my connection (over PPPoE IPv4) but it seems OPNsense is not setting the default route correctly i.e. uses vtnet0_vlan100 instead of pppoe0, and thus traffic fails to leave my firewall.

The gateway is scoped to the parent VLAN but the output interface is a different VLAN:

default  fe80::be24:11ff:fe78:54d5%vtnet0_vlan100  UG  vtnet0_vlan1

Should be:

default  fe80::be24:11ff:fe78:54d5%pppoe0  UG  pppoe0

Impact: LAN clients get SLAAC addresses and default routes from radvd but packets are dropped at the firewall ("No route" ICMPv6).

Workaround:

route -6 delete default
route -6 add default fe80::be24:11ff:fe78:54d5%pppoe0

• UI reload of WAN does NOT fix it (re-triggers the bad route even when removed using the workaround)
• net.inet6.ip6.forwarding is 1
• radvd config is correct, RAs are being sent
• IPv4 over the same PPPoE link works fine

Possibly related: https://github.com/opnsense/core/issues/2186 but was closed due to inactivity

For now I can utilise a cron job to fix it, but is there another way I'm missing?

[franco]

Hi!

What is in:

# ifctl -6r -i pppoe0

And which version are you using?


Cheers,
Franco

[Taomyn]

Hi!

What is in:

# ifctl -6r -i pppoe0

And which version are you using?


Cheers,
Franco

It shows

ifctl -6r -i pppoe0
fe80::b614:89ff:fe08:5d00

I'm on 26.7.8_1-amd64

[franco]

That's strange. The default route should come from that file.


Cheers,
Franco

[franco]

PS: The newwanipv6 logs may help here.

[Taomyn]

PS: The newwanipv6 logs may help here.

Does this help? I used the UI to reload the WAN which breaks the gateway, then I ran my cron job to fix it, then I exported the system log, set to "notice" for that period.

https://pastebin.com/z5ZjVAHA

If not, let me know what you would like me to get.

[franco]

Well these pop out immediately:

> Sending Information Request on vtnet0_vlan250
> Sending Information Request on vtnet0_vlan100

What's their purpose? There could be bugs regarding this. I've never seen anyone using this feature in the last decade. Functionally it's not supposed to do anything other than fetching nameserver information.

> /usr/local/etc/rc.newwanipv6: IP renewal starting (reason: request, address: fe80::be24:11ff:fe78:54d5%pppoe0, interface: wan, device: pppoe0)

This looks good...

> Bypassing address release because of -n flag

This is a dangerous setting from a long time ago.

> /usr/local/etc/rc.newwanipv6: ROUTING: entering configure using wan, lan

Routing for IPv6 never produces any default route, which means FreeBSD decides what the default route is based on the RAs it sees, likely from one of the INFOREQ calls.  Why WAN doesn't even bother adding an IPv6 default route I'm not sure of.


Cheers,
Franco

[Taomyn]

Well these pop out immediately:
> Sending Information Request on vtnet0_vlan250
> Sending Information Request on vtnet0_vlan100
What's their purpose? There could be bugs regarding this. I've never seen anyone using this feature in the last decade. Functionally it's not supposed to do anything other than fetching nameserver information.
> /usr/local/etc/rc.newwanipv6: IP renewal starting (reason: request, address: fe80::be24:11ff:fe78:54d5%pppoe0, interface: wan, device: pppoe0)
This looks good...
> Bypassing address release because of -n flag
This is a dangerous setting from a long time ago.
> /usr/local/etc/rc.newwanipv6: ROUTING: entering configure using wan, lan

Routing for IPv6 never produces any default route, which means FreeBSD decides what the default route is based on the RAs it sees, likely from one of the INFOREQ calls.  Why WAN doesn't even bother adding an IPv6 default route I'm not sure of.
Cheers,
Franco
Cheers,
Franco

For vtnet0_vlan100 + vtnet0_vlan250 are just two VLANs on my local network configured in Interfaces: Devices: VLAN, they're quite old although I no longer use vlan250 so I could remove that at some point, and vlan100 is used by one of my WiFi APs to take guest traffic direct to the Internet.

For the -n I have no idea about this, where would this be coming from and what should I do?

It also seems that my workaround has stopped working or my ISP has made another change and broken it, so now I need to re-do my diagnosis again.

[Taomyn]

Franco, in diagnosing my fix not working I managed to properly sort it, well at least until it breaks again ;-) So for now I don't need my cron "fix" as it's not an issue even on restarts and reconnections.

The final pieces on the puzzle included fixing the Router Advertisment and Kea DHCPv6 enabling to give clients correct IPv6 information. I also hit bug https://github.com/opnsense/core/issues/10906

Nothing explains why it seemed to get fixed before and then stop working, but I'm happy that it's ok for now.

[franco]

Ok so far so good.  The -n option for dhcp6c is at "Interfaces: Settings: Prevent Release" which is an advanced setting. Try disabling this to see if it still works as expected.  It needs a reboot to take effect.


Cheers,
Franco

[Taomyn]

Ok so far so good.  The -n option for dhcp6c is at "Interfaces: Settings: Prevent Release" which is an advanced setting. Try disabling this to see if it still works as expected.  It needs a reboot to take effect.


Cheers,
Franco

I disabled that as well now, rebooted and so far still working - no idea why that was enabled, maybe left over from the last time I tried to get IPv6 working a few years back.

[franco]

Yup it used to be a bit more effective with janky PPPoE behaviour some years ago, but it has gotten better so it's not strictly needed anymore (and not set by default either).


Cheers,
Franco