IPv6 from Android devices still broken

[rmayr]

Unfortunately, https://forum.opnsense.org/index.php?msg=262791 seems to apply to 26.1 as much as to the 25 series. I have now tried to switch from dnsmasq (which has another bug with SAs, to be reported separately) back to radvd and KEA for DHCPv6, plugging in a carp.d hook script to only let radvd run on the respective master.

This seems stable for my desktop devices, but Android devices, though they consistently get a SLAAC pair of addresses, fail to connect.

Is https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280701 supposed to apply to 26.1 kernels? Has the whole patch set been reverted or is it supposed to be fixed?

[franco]

> Is https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280701 supposed to apply to 26.1 kernels? Has the whole patch set been reverted or is it supposed to be fixed?

We do not really know and they made it clear they really do not care.

I still think FreeBSD users suffer more from this than OPNsense users ever did.


Cheers,
Franco

[OPNenthu]

We do not really know and they made it clear they really do not care.

I'm sure you don't say that lightly but it reads a bit nonchalantly in text.  Is there reassurance for users w.r.t upstream support?  😅

[franco]

> Is there reassurance for users w.r.t upstream support?

Basically all of upstream support is hinged in the fact that nobody is obligated to give upstream support.

They made this VERY clear and it's their and their user's loss.

https://forum.opnsense.org/index.php?topic=50102.0


Cheers,
Franco

[Patrick M. Hausen]

@OPNenthu you can try to open another issue on the FreeBSD bug tracker. For that to have any chance of success, though, you would need to reproduce the issue on a plain FreeBSD system without any additions, specifically not OPNsense.

[franco]

Basically what FreeBSD said the last time this went sideways and then ignored a report by a user for months. I simply have my doubts this will improve by asking more of users yet again.


Cheers,
Franco

[rmayr]

Thanks, @franco for the additional context. I have to admit that wading through the FreeBSD bug report linked above was ... painful ... and does not inspire a lot of confidence in future upstream behavior.

While I could probably try to reproduce on a standard FreeBSD kernel (assuming I could just take the OPNsense generated radvd and pf configs and run them on stock FreeBSD), it would be a major time investment for something that has been bugging me for nearly a year but hasn't been a major personal blocker. And reading the bug report, it might be wasted time anyway if upstream doesn't seem to care much about IPv6 stability with pf. With my current home setup (assuming it doesn't turn up any other unexpected surprises), I can fully use IPv6 locally (ULA) and globally (PD assignment to OPNsense, identity association for VLAN interfaces, radvd with hacky master/backup script, kea) from my Linux desktops/laptops and locally run services. Android clients for now fall back to IPv4-only, which is not terrible and - importantly - is not currently noticed by other members of the household, but still bugs my sense of proper networking setup. If I get another long evening to dedicate to this, I might write up a small post on the setup and all the components coming together.

However, for our university student lab setup, this might become a major blocker for some lab scenarios - especially as the packet dropping is erratic and unpredictable and fairly impossible to debug for students. We'll have to think about some of those and how best to deal with IPv6 in teaching setups. We've been using OPNsense as our main example in student labs due to the great UI and nice integration of components (thanks to the whole team for all the work!) for years, and I'd hate to have to move some of them over to VyOS or OpenWRT. The students would hate the change even more, I'm sure :)

[OPNenthu]

@patrick @franco Thanks- I picked up on another subtle hint into the upstream mindset in a video I was watching to learn about packet flow through the stack.  At the segment starting 00:03:40, there's a comment about the "support policy" if multiple firewalls are enabled.  I guess that could be aimed at OPNsense since we use ipfw for shaping.

@rmayr Although I haven't noticed our Androids not able to get online, I do now wonder if they're falling back to IPv4 and I'm just not aware.  I see F/W and Unbound logs where sometimes the v4 address is used and sometimes the v6, but I thought it might be an artifact of Happy Eyeballs.  Something to dig into at some point, but as you say, no one in the house is complaining.

As a learner of sorts, I totally agree OPNsense is an excellent learning platform.

[franco]

Basically we use https://github.com/opnsense/src/commit/cd9f3dd75849f to match OpenBSD after a number of incomplete backports from OpenBSD to FreeBSD.

Internally, someone from FreeBSD said to me this is not considered correct so it's "naturally" refused. It's also a bit confusing since the initial security report that started this downward trend was only for FreeBSD and not for OpenBSD and not even all OpenBSD commits are equal in this if you follow the FreeBSD argument.

Now, I do not know if OPNsense has all corner cases covered (and OpenBSD for that matter), but I do agree that going back to the drawing board to report this properly and fix it is a huge time sink and unclear if someone even acts on this data afterwards.

One way to "fix" this for educational purposes is to turn off state tracking for the ICMP rules for better or worse, but basically just going back to where the code was before all of this started and try statetype set to "sloppy" or "none" to see if this improves the behaviour:

https://github.com/opnsense/core/blob/5b07e0917484b90d0e9411c5e2c4f8ed5a07b8c7/src/etc/inc/filter.lib.inc#L242

We can consider making this configurable if it has a real world benefit.

"Others" have theorised the solution lies in sysctl knobs as you can see from https://redmine.pfsense.org/issues/16146 but it really all boils down to be careful what you sponsored for.  ;)


Cheers,
Franco

[OPNenthu]

There were previously (maybe ongoing?) issues with IPv6 fragmentation as well, and that video also briefly touches on some of the challenges and work done in FreeBSD to support that for pf.  The gist is that IPv6 packets have to be collected and assembled before passing on to pf in order to assure correct inspection and state tracking (and to avoid potential attacks).  That effort apparently wasn't smooth by the author's own admission.  I don't fault him but I worry about the state of things because we have several GH issues in OPNsense with related observations around icmpv6 and packet loss.

I risk conflating unrelated things, though.  </>

[franco]

I suppose we'll see soon enough how FreeBSD 15.1 performs in OPNsense 26.7.


Cheers,
Franco

[glasi]

This seems stable for my desktop devices, but Android devices, though they consistently get a SLAAC pair of addresses, fail to connect.

Are you sure it hasn't to do with Android's energy saving behaviour? Android >=15 ignores IPv6 if RA lifetime is below a certain threshold (e.g. 180 seconds).