WAN DHCP6 - Odd Gateway Issue

[iamperson347]

Hello,

I have some odd behavior on my opnsense instance when using DHCP6 on my WAN interface. On initial boot, the DHCP6 gateway reports being down. I am am able to resolve the issue by disabling the DHCP6 WAN gateway and then re-enable. However, now that I have been working with openvpn, it causes the same issue with the WAN DHCP6 interface. If I enable my openvpn client in opnsense, the DHCP6 gateway reports down again until I disable and then re-enable the DHCP6 gateway. This also causes a constant disconnect/reconnect loop on the openvpn client until I manually disable/re-enable the DHCP6 gateway.

Any ideas or any logs I can check to find more details?

[marjohn56]

The gateway monitor reports down?


Is it Apinger or Dpinger doing the monitoring?

[iamperson347]

Correct. I'm using dpinger. At first I just thought it would cause inaccurate monitoring results but it appears to affect my openvpn client connectivity as well.

[marjohn56]

Strange.. I'm using dpinger with zero issues and OpenVPN too.


When the gateway reports down, can you still connect to an IPv6 test site?

[marjohn56]

Should have asked, what does the gateways log file show?

[iamperson347]

I just tested connecting to google via a windows client (ping -6 google.com) and that seems fine.  ipv6-test.com reports ok as well.

Here is a snippet from the log. As an asside, why does the ipv6 gateway get assigned to a link local address?

Code: [Select]

Aug 27 22:22:46 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:45 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:44 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:43 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:42 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:41 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:40 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:39 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:38 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:37 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:36 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:35 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:34 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:33 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:32 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:31 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:30 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:29 dpinger: WAN_DHCP6 fe80::868a:8dff:fef0:3419%vtnet1: sendto error: 65
Aug 27 22:22:28 dpinger: GATEWAY ALARM: WAN_DHCP6 (Addr: fe80::868a:8dff:fef0:3419%vtnet1 Alarm: 1 RTT: 0ms RTTd: 0ms Loss: 100%)


[iamperson347]

In the general log files, I see this (this was after re-enabling the openvpn client):

Code: [Select]

Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv4 default gateway set, assuming wan
Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to wan
Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt3'
Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: On (IP address: x.x.x.x) (interface: VPN_WAN[opt3]) (real interface: ovpnc1).
Aug 27 22:26:55 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Aug 27 22:26:54 kernel: ovpnc1: link state changed to UP
Aug 27 22:26:52 kernel: ovpnc1: link state changed to DOWN
Aug 27 22:26:52 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: Resync client1
Aug 27 22:26:52 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP6.
Aug 27 22:26:50 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Aug 27 22:26:50 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Aug 27 22:26:50 opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv4 default gateway set, assuming wan
Aug 27 22:26:50 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to wan
Aug 27 22:26:50 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt3'
Aug 27 22:26:49 opnsense: /usr/local/etc/rc.newwanip: On (IP address: x.x.x.x) (interface: VPN_WAN[opt3]) (real interface: ovpnc1).
Aug 27 22:26:49 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Aug 27 22:26:49 kernel: ovpnc1: link state changed to UP
Aug 27 22:26:47 kernel: ovpnc1: link state changed to DOWN
Aug 27 22:26:47 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: Resync client1
Aug 27 22:26:47 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP6.
Aug 27 22:26:45 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Aug 27 22:26:45 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Aug 27 22:26:45 opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv4 default gateway set, assuming wan
Aug 27 22:26:45 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to wan
Aug 27 22:26:45 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt3'
Aug 27 22:26:44 opnsense: /usr/local/etc/rc.newwanip: On (IP address: x.x.x.x) (interface: VPN_WAN[opt3]) (real interface: ovpnc1).
Aug 27 22:26:44 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Aug 27 22:26:44 kernel: ovpnc1: link state changed to UP
Aug 27 22:26:42 kernel: ovpnc1: link state changed to DOWN
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: keeping current default gateway 'fe80::868a:8dff:fef0:3419%vtnet1'
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: setting IPv6 default route to fe80::868a:8dff:fef0:3419
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: keeping current default gateway 'x.x.x.x'
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: setting IPv4 default route to x.x.x.x
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: no IPv4 default gateway set, assuming wan
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: IPv6 default gateway set to wan
Aug 27 22:26:14 opnsense: /system_gateways.php: ROUTING: entering configure using defaults
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: keeping current default gateway 'fe80::868a:8dff:fef0:3419%vtnet1'
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: setting IPv6 default route to fe80::868a:8dff:fef0:3419
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: keeping current default gateway 'x.x.x.x'
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: setting IPv4 default route to x.x.x.x
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: no IPv4 default gateway set, assuming wan
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: IPv6 default gateway set to wan
Aug 27 22:26:11 opnsense: /system_gateways.php: ROUTING: entering configure using defaults
Aug 27 22:22:28 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP6.


[marjohn56]

I just tested connecting to google via a windows client (ping -6 google.com) and that seems fine.  ipv6-test.com reports ok as well.

Here is a snippet from the log. As an asside, why does the ipv6 gateway get assigned to a link local address

It's quite normal for a link-local do be used when using dhcp6 on the WAN, however dpinger not recovering is not. What happens if you physically disconnect/reconnect the WAN?

[iamperson347]

Sorry for the delay. I went to test this today and noticed that the WAN DHCP6 gateway was marked down prior to testing. IPV6 connectivity was still ok. I tried to check the logs but I think I lost the window (since it has been a couple of days since looking at this). I disabled and then re-enabled the gateway for the cable disconnect test.

When unplugging the cable for WAN, both WAN gateways went down (ipv4 and ipv6) and then came back up normally (after some delay).

I should mention that my opnsense instance is virtualized with proxmox. However, based on the cable test, it seems the virtualized NIC is working fine.

[marjohn56]

OK.. You can look at all the logs by going to /var/log and reading them. Have a look at system and gateways logs. Something must give us a clue. 

[iamperson347]

It appears it might be some sort of strange interaction with the ipv6 gateway for my ISP. I switched do a different monitoring ip on the gateways and things seems to be stable. I can also disable and re-enable the openvpn client and the gateways monitoring my ISP connection still report up as expected. I wish I had more info as to what is happening with the ISP gateway on ipv6, but that's not a discussion for this forum.