OPNsense Forum

Archive => 21.1 Legacy Series => Topic started by: RobLatour on June 26, 2021, 08:41:49 PM

Title: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 26, 2021, 08:41:49 PM
I just applied the latest update as follows:

2021-06-26T14:26:35   pkg-static[7325]   os-ntopng-enterprise upgraded: 4.3.210622 -> 4.3.210626   
2021-06-26T14:26:28   pkg-static[7325]   ntopng upgraded: 4.3.210622 -> 4.3.210626

and now the DHCPv6 Server is not starting (where it was before the update).

When I check for updates the system says there are no more.

I tried rebooting the machine, still no love.

Any ideas on what is required to get this working again?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 27, 2021, 02:06:31 PM
Awoke to another set of updates this morning, I applied them, but the DHCPv6 Server is still not starting.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 28, 2021, 11:05:54 PM
Are you using a delegated prefix received from the ISP, or are you running a static DHCPv6 address range for the LAN clients?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 28, 2021, 11:22:47 PM
My network's external IP address is assigned by the ISP; as far as I can see it doesn't change very often but does change - if that is what your asking?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 28, 2021, 11:29:00 PM
If you try to start the service, does it stay running? Is IPv6 currently working on the network?

My network uses a delegated prefix from the ISP and I use a separate monitoring IP for the dhcpv6 "status" due to some weirdness with dhcp6c in BSD.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 28, 2021, 11:45:16 PM
If I try to start the service, it will not start.

when I visit https://test-ipv6.com/  this is what I get:

Test with IPv4 DNS record -    
ok (0.105s) using ipv4

Test with IPv6 DNS record       
bad (0.007s)

Test with Dual Stack DNS record       
ok (0.107s) using ipv4

Test for Dual Stack DNS and large packet       
ok (0.100s) using ipv4

Test IPv6 large packet       
bad (0.005s)

Test if your ISP's DNS server uses IPv6       
ok (0.101s) using ipv4

Find IPv4 Service Provider       
ok (0.106s) using ipv4 ASN 812

Find IPv6 Service Provider       
bad (0.006s)


When I sign on to my router, the only thing I can see related to ipv4 vs ipv6 is

Router Mode - and its set to dual (meaning both ipv4 and ipv6).
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 28, 2021, 11:52:25 PM
Also, for testing purposes, I just plugged my pc into my router directly (bypassing my opnsense box).

Here are the differening results when I visit https://test-ipv6.com/ bypassing the opnsense box:

Test with IPv4 DNS record       
ok (0.362s) using ipv4

Test with IPv6 DNS record       
ok (0.399s) using ipv6

Test with Dual Stack DNS record       
ok (0.423s) using ipv6

Test for Dual Stack DNS and large packet       
ok (0.363s) using ipv6

Test IPv6 large packet       
ok (0.594s) using ipv6

Test if your ISP's DNS server uses IPv6       
ok (0.516s) using ipv6

Find IPv4 Service Provider       
ok (0.387s) using ipv4 ASN 812

Find IPv6 Service Provider       
ok (0.412s) using ipv6 ASN 812
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 28, 2021, 11:56:31 PM
Ah, so there's another router upstream from OPNsense? Within the OPNsense UI, if you go to Interfaces/Overview, do you see IPv6 addresses present on the WAN and LAN interfaces?

If you do see IPv6 addresses listed, you can go to Interfaces/Diagnostics/Ping and try to run an ipv6 ping to an external source (youtube.com or some other ipv6 enabled domain). Verify that OPNsense can actually ping out on ipv6. If not, then it probably isn't getting an IPv6 address from that upstream router.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 12:22:16 AM
The upstream box, is my ISP's router.  It connects directly to the opnsense box.

As mentioned above, the only reference to an ipv6 on the ISP router's windows is the one that says that is working in dual mode (i.e. supporting both ipv4 and ipv6).

I have now reconnected my computer to the opnsense box.  So ISP router > opnsense box > my computer.

Having done that, as requested I went to http://www.ipv6now.com.au/pingme.php and pinged google.com, here are the results:

The response for 'google.com' using IPv4 is:
PING google.com (172.217.5.110) 56(84) bytes of data.
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=1 ttl=121 time=1.34 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=2 ttl=121 time=1.41 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=3 ttl=121 time=1.44 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=4 ttl=121 time=1.40 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=5 ttl=121 time=1.50 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 1.342/1.422/1.503/0.062 ms

The response for 'google.com' using IPv6 is:
PING google.com(sfo03s18-in-x0e.1e100.net) 56 data bytes
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=1 ttl=121 time=1.49 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=2 ttl=121 time=1.50 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=3 ttl=121 time=1.58 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=4 ttl=121 time=1.50 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=5 ttl=121 time=1.53 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 1.495/1.523/1.587/0.060 ms

Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 12:25:15 AM
also, for clarity this is what I am seeing on my lobby dashboard:
https://ibb.co/5jFRK0D
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 12:31:45 AM
That's a good sign, it looks like an ipv6 prefix is still being delegated to OPNsense, and that is being handed out to clients on the LAN side.

If you're still see a red status for the DHCPv6 server, it's likely due to the gateway monitoring not able to ping the upstream router that is assigning the prefix. Can you try setting a different ipv6 gateway monitor IP as shown in the screenshot and check if the dhcpv6 service will stay started?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 01:00:20 AM
First, thank you for all your help.

Second, I'm not exactly sure how this should be set up; and there is already an entry that exists ...

(I've masked out the actual address in red)

https://ibb.co/vxFGLXy

Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 01:07:58 AM
So right now it seems to be working so I would suggest just waiting and see if the service stays online.

However, if it goes back offline after some time, what I have seen in the past is that some configurations need to use an external IP to ping and keep the gateway status 'online'. If you find that this goes offline after some time, you can click the edit button on the DHCP6 gateway and specify a different ipv6 address as shown in the screenshot. I've found that this helps stabilize the status of the dhcp6 service and it's an easily reversible change if it doesn't end up working for your environment.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 01:44:45 AM
I guess I still don't get it. 

If it is working why is the lobby dashboard still showing a red square instead of a green triangle?

Also, as noted above, https://test-ipv6.com/ only works for ipv6 when the opnsense box is taken out of the equation.

Regarding,  ... specify a different ipv6 address as shown in the screenshot ...
a specific address is not entered - it says dynamic on the opnsense window as shown here:
https://ibb.co/cQp6yLQ

If I need to enter a specific address, where would I get that from?

Again, thank you for your help.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 02:26:40 AM
Quote from: RobLatour on June 29, 2021, 01:44:45 AM
Also, as noted above, https://test-ipv6.com/ only works for ipv6 when the opnsense box is taken out of the equation.

Quote from: RobLatour on June 29, 2021, 12:22:16 AM
The upstream box, is my ISP's router.  It connects directly to the opnsense box.

As mentioned above, the only reference to an ipv6 on the ISP router's windows is the one that says that is working in dual mode (i.e. supporting both ipv4 and ipv6).

I have now reconnected my computer to the opnsense box.  So ISP router > opnsense box > my computer.

Having done that, as requested I went to http://www.ipv6now.com.au/pingme.php and pinged google.com, here are the results:

The response for 'google.com' using IPv4 is:
PING google.com (172.217.5.110) 56(84) bytes of data.
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=1 ttl=121 time=1.34 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=2 ttl=121 time=1.41 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=3 ttl=121 time=1.44 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=4 ttl=121 time=1.40 ms
64 bytes from sfo03s07-in-f110.1e100.net (172.217.5.110): icmp_seq=5 ttl=121 time=1.50 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 1.342/1.422/1.503/0.062 ms

The response for 'google.com' using IPv6 is:
PING google.com(sfo03s18-in-x0e.1e100.net) 56 data bytes
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=1 ttl=121 time=1.49 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=2 ttl=121 time=1.50 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=3 ttl=121 time=1.58 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=4 ttl=121 time=1.50 ms
64 bytes from sfo03s18-in-x0e.1e100.net: icmp_seq=5 ttl=121 time=1.53 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 1.495/1.523/1.587/0.060 ms



I'm confused by these two? I thought you said you plugged OPNsense back in and the PC behind OPNsense was able to ping both IPv4 and IPv6 addresses?

Can you login to OPNsense and navigate to Interfaces/Overview on the left hand side of the screen. Then expand the WAN and LAN interfaces. Do you see an IPv6 address listed on those interfaces?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 02:52:21 AM
There is an entry for the lan, and for wan here is what I see:
https://ibb.co/tXX0SbZ

I notice there is yet another update available for opnsense ; should I apply it?

Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 02:57:43 AM
Hmm, it seems odd that there is a /128 and a /64 assigned to WAN.

Can you post a screenshot of the Interfaces/WAN configuration page? Specifically the Generic Configuration section and the DHCPv6 Client Config section on that page.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 03:32:18 AM
here is the whole thing:
https://ibb.co/QNmgQy3
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 03:37:47 AM
also I just applied the latest update and rebooted.  It was of no help, the same problem remains.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 04:48:50 AM
Within the UI, if you screenshot the output of Interfaces/Overview/LAN, does the LAN interface also show an ipv6 address?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 01:05:51 PM
Yes, it does.

https://ibb.co/Fs9xW5B

also, for ipv6 on this window I just tried a 'release' followed by a 'renew', but that did not help either.

Addtionally, I applied another set of updates from opnsense this morning, again with no change.

Is there a log someplace that would describe why the lobby's dashboard would be reporting red for the DHCPv6 Server even after I try to start it?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 03:25:53 PM
That last screenshot actually doesn't show a prefix being delegated to the LAN interface. That interface is only showing a link-local address space.

At this point I would enable ipv6 debug logging and see if that gives any clues. However, I think the main source is the use of the Rogers supplied router.

To enable ipv6 debug logging: Interfaces/Settings/IPv6DHCP and change the log level drop down menu to 'debug'

You could also try some of the settings listed here, use the pfSense recommended settings. It appears that quite a few Rogers customers had issues with IPv6 stability. https://communityforums.rogers.com/t5/Internet/Rogers-IPv6-Status/m-p/373238/highlight/true#M36710
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: franco on June 29, 2021, 04:08:59 PM
Quoting https://communityforums.rogers.com/t5/Internet/Rogers-IPv6-Status/m-p/373238/highlight/true#M36710

> DHCPv6 Prefix Delegation Size: 64

WTF?

Apparently they messed with the prefix delegation and the coincidental reboot caused the prefix to disappear. There's nothing in the update(s) that would cause this.


Cheers,
Franco
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 04:11:25 PM
I enabled the debug logging, tried another restart, and even boot, and looked for the results in system - logfiles - general (which I assume is the correct place) but didn't see any errors or warnings.

While I am not fully discounting the fact the issue could be with the Rogers, the DHCPv6 Server had been working fine up until I applied the most recent upgrade four or five days ago.

Is there a way to roll back to the prior version?
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 04:13:55 PM
(Franko) if by 'they' you mean me, I didn't mess with anything - the service just stopped working when I applied the update.

Is there something I can manually do to fix this up - its all well over my level of expertise :-)
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: franco on June 29, 2021, 04:29:12 PM
No, sorry, I meant the ISP as per that link. Take a look at the pfSense settings at the bottom and try them out on your install:

Use IPv4 connectivity as parent interface: yes
Request only a IPv6 prefix: no
DHCPv6 Prefix Delegation Size: 64
Send IPv6 prefix hint: yes


It supposedly sounds like they use PPPoE on IPv4? These settings are weird... I know that "Request only a IPv6 prefix: no" can cause a lot of issues with tight delegation where you end up with the same network on WAN and LAN and that can't possibly route correctly.


Cheers,
Franco
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 05:41:55 PM
Thanks.

On the Interfaces - [WAN_Rogers] window

I checked 'Use IPv4 connectivity" (it was previously unchecked)
I left "Request only a IPv6 prefix" unchecked
I left "DHCPv6 Prefix Delegation Size" set at 64
I checked 'Send IPv6 prefix hint" (it was previously unchecked)

saved and applied the changes.

However, sadly, the DHCPv6 Server will still not start.
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: opnfwb on June 29, 2021, 08:20:18 PM
At this point you'll need to check the logs and see if there are more details.

To enable ipv6 debug logging: Interfaces/Settings/IPv6DHCP and change the log level drop down menu to 'debug'
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 08:47:13 PM
Here is what is in the log after I click the start service button on the lobby/dashboard screen:

https://ibb.co/8rw1dx0
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 09:12:20 PM
It's working!

There is a setting on my Rogers router that says "Residential Gateway Function" which I changed from Enabled to Disabled.

After that, and a reboot of the router, the opnsense DHCPv6 Server was able to start.

Unfortunately, now I can't seem to sign on to my Rogers Router any more - so I will have to figure that one out.

I will also now try and revert back the new settings I had made to the opnsense box, and will post back here to confirm if they were needed or not - not that the "Residential Gateway Function" is disabled.

Thank you so much for your time and help opnfwb and franco!
Title: Re: DHCPv6 Server not starting following most recent firmware update
Post by: RobLatour on June 29, 2021, 09:33:13 PM
In terms of reverting settings,
I unchecked 'Use IPv4 connectivity" and 'Send IPv6 prefix hint"
applied the changes and rebooted.

With these changes, the DHCPv6 Server continued to work just fine.

Again, with thanks for your help opnfwb and franco!