PSA: recent Comcast firmware seems to break prefix delegation for static IPv6

[really_lost]

I lost my IPv6 prefix delegation about a week ago. Seriously dug into it yesterday and have packet captures of the modem telling me no prefix delegation.

For anyone else using anything besides the base /64 of your IPv6 statics, don't waste much time on this. There's a forum thread on the Comcast support that makes it clear this is an issue with the latest firmware. It even includes someone who got their modem swapped out. The swapped out modem came with older firmware. Prefix delegation worked again. A few days later, the new modem switched to the latest firmware and prefix delegation broke.

https://forums.businesshelp.comcast.com/conversations/ipv6/prefix-delegation-disabled/690fa973a2c50219bf21c6e6

It's pretty clear that firmware CGA4332COM_8.2p5s1_PROD_sey breaks prefix delegation for Comcast customers.

[really_lost]

I was ultimately able to get Comcast to roll back my firmware and prefix delegation is working again. It took about a week, but I also opened the case the Wednesday before Thanksgiving.

Comcast is aware of the issue. They will eventually roll out updated firmware, but it sounds like that will take time. If you are affected by this, you'll want to get a ticket opened and request a firmware rollback.

[OPNenthu]

Interesting... looks like a recent firmware update brought some changes to residential modems, too (screen attached).

[franco]

"allan" who reported the issue in the Comcast forum is an OPNsense user so maybe he has an update on the situation for us?


Cheers,
Franco

[allan]

Thanks for telling me about this thread, Franco. I spoke to someone in their corporate escalations group on Nov 10. Even he had to find a way to get it escalated into their engineering group. By Dec 1st, they rolled back my firmware at my request and I confirmed that fixed the problem (again). They then started rolling everyone back on Dec 5th and expected to complete that process by Dec 8th. He was going to update me if things change and I was going to reach out if the rollback caused issues. Thankfully, all went well.

Sadly, this is not the first time firmware updates affected my IPv6. My previous event triggered the modem's firewall and *block all incoming IPv6 connections* even though it is set to "disabled". Port forwarding, IPSec, client VPNs all went down. Similar to this time, I found someone who was able to relay it into engineering.

Btw, the one I am eagerly awaiting news on is the CheckPoint vs StrongSwan 6.0.3 CHILD_CREATE issue we had (#9382). The latest info I received today was their R&D discussed my case in their meeting and they will investigate my issue before making a decision. I set up a lab to gather logs and sent it all in along with Tobias' comments and links to the RFCs. I hope it was convincing enough.

[allan]

If you are affected by this, you'll want to get a ticket opened and request a firmware rollback.

I really want to emphasize this to anyone reading. IPv6-PD is not commonly used and it is not actively monitored-at least by Tier 1 support since they told me their diagnostics all show green. It takes everyone affected to call and open a ticket before someone notices. The call volume has to be enough to show up in their reports. Otherwise, our issue stays below their radar and they consider us "isolated issues".

[OPNenthu]

IPv6-PD is not commonly used and it is not actively monitored-at least by Tier 1 support since they told me their diagnostics all show green.

If that's the case for business accounts... then the fact that IPv6-PD works at all for my home connection is something of a miracle and I'm on my own.

Great.