Strange IPv6 behavior after update

[mahescho]

Hi Franco,

just let me know what you need 

Matthias

[Droppie391]

Hi Franco,

i installed the patch y´day but when the interface did its reset this morning, the ipv6 default route returned to wan again. i had to set it to pppoe on the console to get ipv6 internet working again.

[franco]

@Matthias

Do you have a "Reload" button under Interfaces: Overview: WAN ? If so, can you try this one instead...

@ Droppie391

Do you have the system logs for this incident? Search for "ROUTING".


Thanks,
Franco

[Droppie391]

i will post the log eintries later but found something more disturbing, it looks like the pppoe interface has linked to the wrong hw interface.:

igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=4400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
   ether 78:24:af:82:a2:28
   hwaddr 78:24:af:82:a2:28
   inet6 fe80::7a24:afff:fe82:a228%igb0 prefixlen 64 scopeid 0x1
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=4400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
   ether 78:24:af:82:a2:29
   hwaddr 78:24:af:82:a2:29
   inet6 fe80::7a24:afff:fe82:a229%igb1 prefixlen 64 scopeid 0x2
pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
   inet6 fe80::7a24:afff:fe82:a229%pppoe0 prefixlen 64 scopeid 0x11

igb0 is our wan link, igb1 is our lan interface
in the point2point settings it is set to link to igb0 (see attached image)

so it looks like the routing isn´t the problem at all....

[Droppie391]

and here the log-extract (all IP addresses etc masked)
the log looks exactly the same, before and after applying the patch

Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: Accept router advertisements on interface igb0
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: The command '/var/etc/rtsold_igb0_script.sh' returned exit code '1', the output was ''
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: entering configure using 'wan'
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: IPv4 default gateway set to wan
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: IPv6 default gateway set to wan
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: setting IPv4 default route to ISP_IPV4_GW
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: removing /tmp/pppoe0_defaultgw
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: creating /tmp/pppoe0_defaultgw using 'ISP_IPV4_GW'
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: The command '/sbin/route add -'inet' default 'ISP_IPV4_GW'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add net default: gateway ISP_IPV4_GW fib 0: Network is unreachable'
Sep 12 05:00:00 my_hostname opnsense: /usr/local/etc/rc.configure_interface: ROUTING: skipping IPv6 default route
Sep 12 05:00:03 my_hostname opnsense: /usr/local/etc/rc.configure_interface: The command `/sbin/ifconfig -m 'pppoe0'' failed to execute ifconfig: interface pppoe0 does not exist
Sep 12 05:00:04 my_hostname kernel: ng0: changing name to 'pppoe0'
Sep 12 05:00:04 my_hostname opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'pppoe0'
Sep 12 05:00:04 my_hostname opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::aaaa:bbff:fe82:a228) (interface: MY_WAN_ALIAS[wan]) (real interface: pppoe0).
Sep 12 05:00:05 my_hostname opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'pppoe0'
Sep 12 05:00:05 my_hostname opnsense: /usr/local/etc/rc.newwanip: On (IP address: MY_IPV4_GLOBAL_IP) (interface: MY_WAN_ALIAS[wan]) (real interface: pppoe0).
Sep 12 05:00:06 my_hostname opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Sep 12 05:00:06 my_hostname opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan
Sep 12 05:00:06 my_hostname opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to wan
Sep 12 05:00:06 my_hostname opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to ISP_IPV4_GW
Sep 12 05:00:06 my_hostname opnsense: /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway 'ISP_IPV4_GW'
Sep 12 05:00:06 my_hostname opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route

[Droppie391]

here the ppps log for what its worth:

Sep 12 05:00:04 my_hostname ppp: [wan] Bundle: Interface ng0 created
Sep 12 05:00:04 my_hostname ppp: [wan_link0] Link: OPEN event
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: Open event
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: state change Initial --> Starting
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: LayerStart
Sep 12 05:00:04 my_hostname ppp: [wan_link0] PPPoE: Connecting to ''
Sep 12 05:00:04 my_hostname ppp: PPPoE: rec'd ACNAME "isp_pppoe_gw"
Sep 12 05:00:04 my_hostname ppp: [wan_link0] PPPoE: connection successful
Sep 12 05:00:04 my_hostname ppp: [wan_link0] Link: UP event
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: Up event
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: state change Starting --> Req-Sent
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: SendConfigReq #1
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   PROTOCOMP
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MRU 1492
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MAGICNUM 0xdc2033c6
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: rec'd Configure Request #1 (Req-Sent)
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MRU 1492
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   AUTHPROTO CHAP MD5
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MAGICNUM 0xd420a811
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MP MRRU 1524
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   ENDPOINTDISC [LOCAL] 62 72 61 73 66 72 61 34 30 35
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MULTILINKHEADERFMT
Sep 12 05:00:04 my_hostname ppp: [wan_link0]     Not supported
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: SendConfigRej #1
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MP MRRU 1524
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MULTILINKHEADERFMT
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: rec'd Configure Ack #1 (Req-Sent)
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   PROTOCOMP
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MRU 1492
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MAGICNUM 0xdc2033c6
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: state change Req-Sent --> Ack-Rcvd
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: rec'd Configure Request #2 (Ack-Rcvd)
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MRU 1492
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   AUTHPROTO CHAP MD5
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MAGICNUM 0xd420a811
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   ENDPOINTDISC [LOCAL] 62 72 61 73 66 72 61 34 30 35
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: SendConfigAck #2
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MRU 1492
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   AUTHPROTO CHAP MD5
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   MAGICNUM 0xd420a811
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   ENDPOINTDISC [LOCAL] 62 72 61 73 66 72 61 34 30 35
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: state change Ack-Rcvd --> Opened
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: auth: peer wants CHAP, I want nothing
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: LayerUp
Sep 12 05:00:04 my_hostname ppp: [wan_link0] CHAP: rec'd CHALLENGE #1 len: 31
Sep 12 05:00:04 my_hostname ppp: [wan_link0]   Name: "isp_pppoe_gw"
Sep 12 05:00:04 my_hostname ppp: [wan_link0] CHAP: Using authname "my_login_name"
Sep 12 05:00:04 my_hostname ppp: [wan_link0] CHAP: sending RESPONSE #1 len: 40
Sep 12 05:00:04 my_hostname ppp: [wan_link0] CHAP: rec'd SUCCESS #1 len: 4
Sep 12 05:00:04 my_hostname ppp: [wan_link0] LCP: authorization successful
Sep 12 05:00:04 my_hostname ppp: [wan_link0] Link: Matched action 'bundle "wan" ""'
Sep 12 05:00:04 my_hostname ppp: [wan_link0] Link: Join bundle "wan"
Sep 12 05:00:04 my_hostname ppp: [wan] Bundle: Status update: up 1 link, total bandwidth 64000 bps
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: Open event
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: state change Initial --> Starting
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: LayerStart
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: Open event
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: state change Initial --> Starting
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: LayerStart
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: Up event
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: state change Starting --> Req-Sent
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: SendConfigReq #1
Sep 12 05:00:04 my_hostname ppp: [wan]   IPADDR 0.0.0.0
Sep 12 05:00:04 my_hostname ppp: [wan]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: Up event
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: state change Starting --> Req-Sent
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: SendConfigReq #1
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: rec'd Configure Request #1 (Req-Sent)
Sep 12 05:00:04 my_hostname ppp: [wan]   COMPPROTO VJCOMP, 16 comp. channels, allow comp-cid
Sep 12 05:00:04 my_hostname ppp: [wan]   IPADDR ISP_IPV4_GW
Sep 12 05:00:04 my_hostname ppp: [wan]     ISP_IPV4_GW is OK
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: SendConfigAck #1
Sep 12 05:00:04 my_hostname ppp: [wan]   COMPPROTO VJCOMP, 16 comp. channels, allow comp-cid
Sep 12 05:00:04 my_hostname ppp: [wan]   IPADDR ISP_IPV4_GW
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: state change Req-Sent --> Ack-Sent
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: rec'd Configure Request #1 (Req-Sent)
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: SendConfigAck #1
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: state change Req-Sent --> Ack-Sent
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: rec'd Configure Nak #1 (Ack-Sent)
Sep 12 05:00:04 my_hostname ppp: [wan]   IPADDR MY_IPV4_GLOBAL_IP
Sep 12 05:00:04 my_hostname ppp: [wan]     MY_IPV4_GLOBAL_IP is OK
Sep 12 05:00:04 my_hostname ppp: [wan] IPCP: SendConfigReq #2
Sep 12 05:00:04 my_hostname ppp: [wan]   IPADDR MY_IPV4_GLOBAL_IP
Sep 12 05:00:04 my_hostname ppp: [wan]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: rec'd Configure Ack #1 (Ack-Sent)
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: state change Ack-Sent --> Opened
Sep 12 05:00:04 my_hostname ppp: [wan] IPV6CP: LayerUp
Sep 12 05:00:04 my_hostname ppp: [wan]   aaaa:bbff:fe82:a229 -> 0123:56ff:fe89:abcd
Sep 12 05:00:05 my_hostname ppp: [wan] IFACE: Up event
Sep 12 05:00:05 my_hostname ppp: [wan] IFACE: Rename interface ng0 to pppoe0
Sep 12 05:00:05 my_hostname ppp: [wan] IPCP: rec'd Configure Ack #2 (Ack-Sent)
Sep 12 05:00:05 my_hostname ppp: [wan]   IPADDR MY_IPV4_GLOBAL_IP
Sep 12 05:00:05 my_hostname ppp: [wan]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
Sep 12 05:00:05 my_hostname ppp: [wan] IPCP: state change Ack-Sent --> Opened
Sep 12 05:00:05 my_hostname ppp: [wan] IPCP: LayerUp
Sep 12 05:00:05 my_hostname ppp: [wan]   MY_IPV4_GLOBAL_IP -> ISP_IPV4_GW

[mahescho]

@Matthias

Do you have a "Reload" button under Interfaces: Overview: WAN ? If so, can you try this one instead...

There is no "Reload" button ...

[mahescho]

I've disconnected the WAN port an reconnected it with the same result. IPv6 stopped working.

[Droppie391]

concerning my problem, i enabled the "Use IPv4 Connection" option and now it works.
The help text was a bit confusing here, following the code in "function get_real_interface"

                    default:
                        switch ($cfg['ipaddr']) {
                            case 'pppoe':
                            case 'ppp':
                            case 'l2tp':
                            case 'pptp':
                                if (isset($cfg['dhcp6usev4iface'])) {
                                    $wanif = $cfg['if'];
                                } else {
                                    $parents = get_parent_interface($interface);
                                    if (!empty($parents[0])) {
                                        $wanif = $parents[0];
                                    } else {
                                        $wanif = $cfg['if'];
                                    }
                                }
                                break;

where the parent interface is selected in stead of the pppoe interface without the option set, it appears to affect the routing table too.

Problem solved, my case is closed.

[mahescho]

Tested to enable "Use IPv4 Connection". Didn't fix my problem.

[franco]

"Use IPv4 Connection" is not useful if your connection is already working sometimes.

It's really strange to have the firewall toggle to reenable packet flow. This is only for IPv6, right? Do you see blocks in the firewall log when the traffic is "stuck"?

[mahescho]

Just for the record: This still does not work as expected ...

[mahescho]

Solution:

https://github.com/opnsense/core/issues/3015

[mahescho]

Final fix:

https://github.com/opnsense/core/commit/c3213d4d918e0675478feda494dc385ae6ccecd1