Intermittent IPv6 drops

Started by cloudz, August 13, 2024, 10:36:50 AM

Previous topic - Next topic
Since upgrading to 24.7.1 I see intermittent IPv6 drops for a few to 15 seconds. I get alerted each time by Uptime Kuma that a specific server isn't available anymore.

Would this be the same issue as the kernel problems with traceroute / icmp-echo?

It's not just ICMP that fails at those times but also existing SSH connections are dropped.

August 13, 2024, 12:45:13 PM #1 Last Edit: August 13, 2024, 01:31:19 PM by cloudz
Downgraded the kernel to 24.7 seems to have fixed [most of] the issue.
Tried it with a few of the ones mentioned in the traceroute topic but that made everything unstable.
Running on a DEC740

IPv6 sometimes spikes to 100+ seconds but the packets don't get lost.

I too have this problem but in my case it appears to affect only instances that were upgraded to 24.7.

I have a fresh 24.7 instance subsequently upgraded to 24.7.1 that performs perfectly in the same scenario as an upgraded instance that required the kernel roll-back to perform properly.

Hi,

I'm using OPNsense in bridge mode with a Freebox Ultra, which is the router provided by the French ISP Free.fr. Since the update to 24.7.1, and possibly even 24.7, I've noticed issues accessing all IPv6 domains, along with massive ICMP drops. Today, I downgraded the kernel to 24.7, and although ICMP stability has improved (monitored via Smokeping), I'm still experiencing slowdowns and access problems across all IPv6 domains (Google, Facebook, etc.). Even worse, streaming services like OQEE (Free's IPTV service) are completely unusable with constant micro-cuts, especially since this service is only accessible via IPv6.

***GOT REQUEST TO AUDIT HEALTH***
Currently running OPNsense 24.7.1 at Mon Aug 19 09:26:32 CEST 2024
>>> Root file system: /dev/gpt/rootfs
>>> Check installed kernel version
Version 24.7 is incorrect, expected: 24.7.1

You need to revert OPNsense too.
Use the command opnsense-revert on the commandline to go back to 24.7.

For me, it solves most of the issues. I sometimes still see some higher spikes.

I also applied the patch for the health monitoring, that took down the CPU load too.

Quote from: slykens on August 19, 2024, 04:56:54 AM
I too have this problem but in my case it appears to affect only instances that were upgraded to 24.7.

I have a fresh 24.7 instance subsequently upgraded to 24.7.1 that performs perfectly in the same scenario as an upgraded instance that required the kernel roll-back to perform properly.

That is weird -- also a clean config? Or do all devices have a similar config?

Quote from: cloudz on August 19, 2024, 11:47:11 AM
Quote from: slykens on August 19, 2024, 04:56:54 AM
I too have this problem but in my case it appears to affect only instances that were upgraded to 24.7.

I have a fresh 24.7 instance subsequently upgraded to 24.7.1 that performs perfectly in the same scenario as an upgraded instance that required the kernel roll-back to perform properly.

That is weird -- also a clean config? Or do all devices have a similar config?

The 24.7.1 that was a fresh install of 24.7 was a clean config and it does not have problems.

The 24.7.1 that did not work was installed last fall and upgraded as new releases came out.

I can confirm this behavior too just after upgrading from 24.1.10

I have the same setup (freebox + opnsense) and i get IPv6 ICMP packet loss on most website (google, facebook, quad9...)
Using a freebox in bridge mode.

I've yet to downgrade the kernel and Opnsense to 24.7 or 24.1.10.

There is an issue being tracked : https://github.com/opnsense/src/issues/218

Useful to follow that one.

Will do, thanks for the heads up!