IPv6 connectivity loss on 24.1.10

[Ben S]

Hi,

This problem may be related to https://forum.opnsense.org/index.php?topic=41508.0 but my problems appear slightly different.

After the update to 24.1.10 I rebooted and everything appeared fine.  After 30 minutes (first IPv6 renewal) I seemed to loose IPv6 connectivity.  Logs showed dhcp6c sending RENEW and then SOLICIT messages.  When trying to diagnose this I could see the packets were being allowed out - checked with

Code: [Select]

$ sudo tcpdump -ve -i pflog0 udp port 546 or udp port 547
tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
21:37:43.196073 rule 105/0(match) [uid 0]: pass out on igb0: (hlim 1, next-header UDP (17) payload length: 89) fe80::2e0:xxx.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x29e8 -> 0x1be3!] dhcp6 solicit (xid=2feda3 (client-ID hwaddr/time type 1 time xx xx) (elapsed-time 65535) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/56 pltime:4294967295 vltime:4294967295)))
21:39:35.293493 rule 105/0(match) [uid 0]: pass out on igb0: (hlim 1, next-header UDP (17) payload length: 89) fe80::2e0:xxx.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x29e8 -> 0x2881!] dhcp6 solicit (xid=e134 (client-ID hwaddr/time type 1 time xx xx) (elapsed-time 0) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/56 pltime:4294967295 vltime:4294967295)))
21:41:41.236162 rule 103/0(match) [uid 0]: pass out on igb0: (hlim 1, next-header UDP (17) payload length: 89) fe80::2e0:xxx.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x29e8 -> 0x484f!] dhcp6 solicit (xid=419032 (client-ID hwaddr/time type 1 time xx xx) (elapsed-time 12531) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/56 pltime:4294967295 vltime:4294967295)))

But running a similar tcpdump command on igb0 (WAN) did not show the packets actually being sent, despite them showing as 'pass out' in the pf log.  Which I find rather confusing.  I'm not sure if the bad checksum notices in the pflog output are significant.

After rebooting the IPv6 has come back, but I don't yet know if it will stay up.  Reloading the WAN interface from the UI > Interfaces > Overview didn't bring it back.

My IPv6 has been working fine on 24.1.7 and 24.1.9.

Any suggestions would be much appreciated.

Thanks
Ben

[newsense]

Looks like this was expected according to the release notes...

Also, a bad dhcp6c patch has been reverted which requires a manual reboot to take full effect.

[1243]

Nope: "After rebooting the IPv6 has come back, but I don't yet know if it will stay up.  Reloading the WAN interface from the UI > Interfaces > Overview didn't bring it back."

They broke IPv6 with 24.1.9 and have now also broken intra networking with 24.2.10.

[franco]

> They broke IPv6 with 24.1.9 and have now also broken intra networking with 24.2.10.

If only we got more qualified reports like this one IPv6 would work better than ever.

So I have to ask the obvious... what's the actual problem then?


Cheers,
Franco

[1243]

I didn’t want to bring it up before, but you gave me the opening. Franco, look into yourself, and there you will find the problem and the solution.

[franco]

Let me know if I can help or you just want to dis. In the latter case I'll go find someone else to help.


Cheers,
Franco

[Slashing]

Hi there! After upgrade to 24.1.10 and reboot, no prefix. IPv6 address on wan interface is fe80::e63a:6eff:fe60:f068/64 and gateway fe80::21c:73ff:fe00:99
USA, Xfinity

[franco]

thanks, over here we were discussion the filter rule change: https://forum.opnsense.org/index.php?topic=41508.0

If you revert this one https://github.com/opnsense/core/commit/e94baab85 by issuing:

# opnsense-patch e94baab85

If that's the offending one I'm curious how the packet capture looks.


Cheers,
Franco

[Slashing]

Code: [Select]

sudo tcpdump -ve -i pflog0 udp port 546 or udp port 547
Password:
tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
01:31:21.137376 rule 72/0(match): pass in on igc0: (flowlabel 0xe03b8, hlim 1, next-header UDP (17) payload length: 76) fe80::9209:d0ff:fe2c:2bfa.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=7027e2 (client-ID hwaddr type 1 9009d02c2bfa) (option-request DNS-server DNS-search-list) (elapsed-time 65535) (Client-FQDN) (IA_NA IAID:3492555770 T1:3600 T2:5400))
01:31:36.745789 rule 72/0(match): pass in on igc0: (flowlabel 0xd0000, hlim 1, next-header UDP (17) payload length: 58) fe80::1443:eb41:299:bf99.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=cab31f (client-ID hwaddr type 1 7ae0d678f796) (option-request DNS-server DNS-search-list opt_103) (elapsed-time 15768) (IA_NA IAID:0 T1:0 T2:0))
01:32:45.734092 rule 94/0(match) [uid 0]: pass out on igc1: (hlim 1, next-header UDP (17) payload length: 105) fe80::e63a:6eff:fe60:f068.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x4205 -> 0x12a1!] dhcp6 solicit (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (IA_NA IAID:0 T1:0 T2:0) (elapsed-time 25083) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/60 pltime:4294967295 vltime:4294967295)))
01:32:45.812523 rule 17/0(match): block in on igc1: (flowlabel 0xa95d5, hlim 63, next-header UDP (17) payload length: 173) 2001:558:4081:cd::10.dhcpv6-server > fe80::e63a:6eff:fe60:f068.dhcpv6-client: [udp sum ok] dhcp6 advertise (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (server-ID hwaddr/time type 1 time 391757488 14feb5d5b5d0) (IA_NA IAID:0 T1:77079 T2:123327 (IA_ADDR 2001:558:6022:cd:9c37:2b2c:358c:5a3a pltime:154159 vltime:154159)) (IA_PD IAID:0 T1:77079 T2:123327 (IA_PD-prefix 2601:2c1:8e80:7ba0::/60 pltime:154160 vltime:154160)) (DNS-server cdns01.comcast.net cdns02.comcast.net))
01:33:27.347975 rule 72/0(match): pass in on igc0: (flowlabel 0xe03b8, hlim 1, next-header UDP (17) payload length: 76) fe80::9209:d0ff:fe2c:2bfa.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=7027e2 (client-ID hwaddr type 1 9009d02c2bfa) (option-request DNS-server DNS-search-list) (elapsed-time 65535) (Client-FQDN) (IA_NA IAID:3492555770 T1:3600 T2:5400))
01:33:47.010003 rule 72/0(match): pass in on igc0: (flowlabel 0xd0000, hlim 1, next-header UDP (17) payload length: 58) fe80::1443:eb41:299:bf99.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=cab31f (client-ID hwaddr type 1 7ae0d678f796) (option-request DNS-server DNS-search-list opt_103) (elapsed-time 30814) (IA_NA IAID:0 T1:0 T2:0))
01:34:36.785343 rule 94/0(match) [uid 0]: pass out on igc1: (hlim 1, next-header UDP (17) payload length: 105) fe80::e63a:6eff:fe60:f068.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x4205 -> 0xe73f!] dhcp6 solicit (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (IA_NA IAID:0 T1:0 T2:0) (elapsed-time 36188) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/60 pltime:4294967295 vltime:4294967295)))
01:34:36.861682 rule 17/0(match): block in on igc1: (flowlabel 0xa95d5, hlim 63, next-header UDP (17) payload length: 173) 2001:558:4081:cd::10.dhcpv6-server > fe80::e63a:6eff:fe60:f068.dhcpv6-client: [udp sum ok] dhcp6 advertise (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (server-ID hwaddr/time type 1 time 391757488 14feb5d5b5d0) (IA_NA IAID:0 T1:77024 T2:123238 (IA_ADDR 2001:558:6022:cd:9c37:2b2c:358c:5a3a pltime:154048 vltime:154048)) (IA_PD IAID:0 T1:77024 T2:123238 (IA_PD-prefix 2601:2c1:8e80:7ba0::/60 pltime:154049 vltime:154049)) (DNS-server cdns01.comcast.net cdns02.comcast.net))
01:35:34.316332 rule 72/0(match): pass in on igc0: (flowlabel 0xe03b8, hlim 1, next-header UDP (17) payload length: 76) fe80::9209:d0ff:fe2c:2bfa.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=7027e2 (client-ID hwaddr type 1 9009d02c2bfa) (option-request DNS-server DNS-search-list) (elapsed-time 65535) (Client-FQDN) (IA_NA IAID:3492555770 T1:3600 T2:5400))
01:36:37.514103 rule 94/0(match) [uid 0]: pass out on igc1: (hlim 1, next-header UDP (17) payload length: 105) fe80::e63a:6eff:fe60:f068.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x4205 -> 0xb816!] dhcp6 solicit (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (IA_NA IAID:0 T1:0 T2:0) (elapsed-time 48261) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/60 pltime:4294967295 vltime:4294967295)))
01:36:37.588213 rule 17/0(match): block in on igc1: (flowlabel 0xa95d5, hlim 63, next-header UDP (17) payload length: 173) 2001:558:4081:cd::10.dhcpv6-server > fe80::e63a:6eff:fe60:f068.dhcpv6-client: [udp sum ok] dhcp6 advertise (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (server-ID hwaddr/time type 1 time 391757488 14feb5d5b5d0) (IA_NA IAID:0 T1:76963 T2:123141 (IA_ADDR 2001:558:6022:cd:9c37:2b2c:358c:5a3a pltime:153927 vltime:153927)) (IA_PD IAID:0 T1:76963 T2:123141 (IA_PD-prefix 2601:2c1:8e80:7ba0::/60 pltime:153928 vltime:153928)) (DNS-server cdns01.comcast.net cdns02.comcast.net))
01:37:39.979613 rule 72/0(match): pass in on igc0: (flowlabel 0xe03b8, hlim 1, next-header UDP (17) payload length: 76) fe80::9209:d0ff:fe2c:2bfa.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=7027e2 (client-ID hwaddr type 1 9009d02c2bfa) (option-request DNS-server DNS-search-list) (elapsed-time 65535) (Client-FQDN) (IA_NA IAID:3492555770 T1:3600 T2:5400))
01:38:26.031522 rule 94/0(match) [uid 0]: pass out on igc1: (hlim 1, next-header UDP (17) payload length: 105) fe80::e63a:6eff:fe60:f068.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x4205 -> 0x8db2!] dhcp6 solicit (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (IA_NA IAID:0 T1:0 T2:0) (elapsed-time 59113) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/60 pltime:4294967295 vltime:4294967295)))
01:38:26.108827 rule 17/0(match): block in on igc1: (flowlabel 0xa95d5, hlim 63, next-header UDP (17) payload length: 173) 2001:558:4081:cd::10.dhcpv6-server > fe80::e63a:6eff:fe60:f068.dhcpv6-client: [udp sum ok] dhcp6 advertise (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (server-ID hwaddr/time type 1 time 391757488 14feb5d5b5d0) (IA_NA IAID:0 T1:76909 T2:123055 (IA_ADDR 2001:558:6022:cd:9c37:2b2c:358c:5a3a pltime:153819 vltime:153819)) (IA_PD IAID:0 T1:76909 T2:123055 (IA_PD-prefix 2601:2c1:8e80:7ba0::/60 pltime:153820 vltime:153820)) (DNS-server cdns01.comcast.net cdns02.comcast.net))
01:39:36.614890 rule 72/0(match): pass in on igc0: (flowlabel 0xe03b8, hlim 1, next-header UDP (17) payload length: 76) fe80::9209:d0ff:fe2c:2bfa.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=7027e2 (client-ID hwaddr type 1 9009d02c2bfa) (option-request DNS-server DNS-search-list) (elapsed-time 65535) (Client-FQDN) (IA_NA IAID:3492555770 T1:3600 T2:5400))
01:40:17.044694 rule 94/0(match) [uid 0]: pass out on igc1: (hlim 1, next-header UDP (17) payload length: 105) fe80::e63a:6eff:fe60:f068.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x4205 -> 0x749c!] dhcp6 solicit (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (IA_NA IAID:0 T1:0 T2:0) (elapsed-time 65535) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/60 pltime:4294967295 vltime:4294967295)))
01:40:17.116563 rule 17/0(match): block in on igc1: (flowlabel 0xa95d5, hlim 63, next-header UDP (17) payload length: 173) 2001:558:4081:cd::10.dhcpv6-server > fe80::e63a:6eff:fe60:f068.dhcpv6-client: [udp sum ok] dhcp6 advertise (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (server-ID hwaddr/time type 1 time 391757488 14feb5d5b5d0) (IA_NA IAID:0 T1:76854 T2:122966 (IA_ADDR 2001:558:6022:cd:9c37:2b2c:358c:5a3a pltime:153708 vltime:153708)) (IA_PD IAID:0 T1:76854 T2:122966 (IA_PD-prefix 2601:2c1:8e80:7ba0::/60 pltime:153709 vltime:153709)) (DNS-server cdns01.comcast.net cdns02.comcast.net))
01:41:04.726874 rule 72/0(match): pass in on igc0: (flowlabel 0xd0000, hlim 1, next-header UDP (17) payload length: 58) fe80::1443:eb41:299:bf99.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=cab31f (client-ID hwaddr type 1 7ae0d678f796) (option-request DNS-server DNS-search-list opt_103) (elapsed-time 65535) (IA_NA IAID:0 T1:0 T2:0))
01:41:30.673066 rule 72/0(match): pass in on igc0: (flowlabel 0xe03b8, hlim 1, next-header UDP (17) payload length: 76) fe80::9209:d0ff:fe2c:2bfa.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=7027e2 (client-ID hwaddr type 1 9009d02c2bfa) (option-request DNS-server DNS-search-list) (elapsed-time 65535) (Client-FQDN) (IA_NA IAID:3492555770 T1:3600 T2:5400))
01:42:15.044218 rule 94/0(match) [uid 0]: pass out on igc1: (hlim 1, next-header UDP (17) payload length: 105) fe80::e63a:6eff:fe60:f068.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x4205 -> 0x749c!] dhcp6 solicit (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (IA_NA IAID:0 T1:0 T2:0) (elapsed-time 65535) (option-request DNS-server DNS-search-list) (IA_PD IAID:0 T1:0 T2:0 (IA_PD-prefix ::/60 pltime:4294967295 vltime:4294967295)))
01:42:15.119840 rule 17/0(match): block in on igc1: (flowlabel 0xa95d5, hlim 63, next-header UDP (17) payload length: 173) 2001:558:4081:cd::10.dhcpv6-server > fe80::e63a:6eff:fe60:f068.dhcpv6-client: [udp sum ok] dhcp6 advertise (xid=9ed244 (client-ID hwaddr/time type 1 time 758694841 e43a6e60f067) (server-ID hwaddr/time type 1 time 391757488 14feb5d5b5d0) (IA_NA IAID:0 T1:76795 T2:122872 (IA_ADDR 2001:558:6022:cd:9c37:2b2c:358c:5a3a pltime:153590 vltime:153590)) (IA_PD IAID:0 T1:76795 T2:122872 (IA_PD-prefix 2601:2c1:8e80:7ba0::/60 pltime:153591 vltime:153591)) (DNS-server cdns01.comcast.net cdns02.comcast.net))

[Slashing]

after that  opnsense-patch e94baab85 all is ok

[franco]

Ah, very good. The capture is from the bad attempt, right?

[Slashing]

Yes, it from bad attempt

[franco]

Ok how about this then: https://github.com/opnsense/core/commit/0217a1a95b1

# opnsense-revert opnsense
# opnsense-patch 0217a1a95b1


Cheers,
Franco

[Slashing]

Gut! All work. Thank you!
P.S.
After reboot too

[franco]

Thanks, I'll have this hotfixed in a bit.

(see how awesome this teamwork in open source can be)


Cheers,
Franco