HOWTO: Using third-party DHCPv6 servers with FiOS IPv6

[frankhecker]

I have a residential FiOS connection with an OPNsense router directly connected to the FiOS ONT. My goal is to provide IPv6 support on my internal networks using third-party servers for DHCPv6 and DNS over IPv6, completely replacing the OPNsense DNS and DHCP services. (I work for a company that sells commercial DNS/DHCP/IPAM systems, and this serves as my home lab.)

What follows is a description of how I got this working, and in particular how I overcame one hard-to-troubleshoot problem. Since I couldn't find an FAQ or tutorial that addressed my exact situation, I thought this might be of interest to people who also want to deploy independent DHCPv6 servers, especially if they have FiOS.

First, more on the configuration: the small form factor PC I have running OPNsense 23.1.1 has four interfaces, of which I am currently using three: WAN for the FiOS connection, LAN for my main home network, and OPT1 for a second lab network. On the LAN subnet I have two DNS/DHCP servers based on BIND and ISC DHCP, while the OPT1 subnet has a second pair of DNS/DHCP servers based on BIND and Kea. (In case you're curious, these are Infoblox NIOS and BloxOne appliances respectively.)

My first task was getting an IPv6 prefix from FiOS. This is pretty well-explained in a number of places, so I'll just summarize here:

• Go to the "Interfaces" > "WAN" page.
• Under "DHCPv6 client configuration", make the following changes.
• Check the box for "Request only an IPv6 prefix".
• Enter the value "56" for the prefix delegation size.
• Check the box for "Send IPv6 prefix hint". (It's not clear whether this is actually needed, but I set it anyway.)
• Click "Save".

After rebooting the router or otherwise resetting the WAN interface, under "Interfaces" > "Overview" > "WAN" you should see the IPv6 delegated prefix set to something like 2600:40xx:xxyy:yy00::/56. The 2600:4000::/24 prefix appears to be associated with the entire Verizon network, while the 2600:40xx:xx00::/40 prefix appears to be associated with Verizon's local FiOS POP(s). The /40 prefix never changes, but the "yy:yy" part in the /56 prefix will change any time the router is rebooted or the WAN interface is otherwise reset.

This of course means that a third-party DHCPv6 or DNS over IPv6 server will need to be reconfigured every time the FiOS IPv6 /56 prefix changes. (This includes both the IPv6 addresses of the servers themselves and the actual DHCPv6 ranges.) However, a WAN reset doesn't affect internal IPv4 stuff (which is configured using RFC 1918 networks), so I would still have IPv4 service and access to the router and the DHCP/DNS servers to reconfigure the setup. (The DNS/DHCP servers that I'm using can also be reconfigured via an API, so I could script this if I want to.)

The second task is enabling IPv6 on the internal interfaces (LAN and OPT1 in my case) so that they all have separate IPv6 prefixes. Again, this is pretty straightforward, and covered in a variety of places:

• Go to "Interfaces" > "LAN" (or "OPT1", or whatever the name of the appropriate interface is).
• In the section "Track IPv6 Interface", set the IPv6 interface to "WAN" (or whatever the name of your WAN interface is).
• Specify a unique value for the IPv6 prefix ID for each of the internal interfaces that you want IPv6 on. In my case I specified "0" for the first interface (LAN) and "1" for the second interface (OPT1).
• Leave the "manual configuration" checkbox unchecked for now.
• Click "Save".
• Repeat the steps above for each of the interfaces you want to use IPv6 on.

After rebooting or otherwise resetting the WAN interface, under "Interfaces" > "Overview" > "LAN" (or whatever the name of your first interface is) you should see the IPv6 delegated prefix set to something like 2600:40xx:xxyy:yy00::/64. Similarly, under "Interfaces" > "Overview" > "OPT1" (or whatever the name of your second interface is) you should see the IPv6 delegated prefix set to something like 2600:40xx:xxyy:yy01::/64.

The third task is setting up firewall rules to allow IPv6 traffic on the internal networks. (Maybe this is supposed to be done automatically, but in my case I had to do it by hand.)

• Go to "Firewall" > "Rules" > "LAN" (or OPT1, or whatever the name of the appropriate interface is).
• Add a new IPv6 rule to match the existing IPv4 rule. In my case I allowed all IPv4 traffic coming in to the LAN interface to go anywhere.
• Click "+" to add a new rule.
• For the action, select "Pass".
• For the direction, select "in".
• For the TCP/IP version, select "IPv6".
• For the source, select "LAN net" (or "OPT1 net", or whatever).
• For the destination and destination port range, select "any".
• Click "Save".
• Repeat the steps above for each of the internal interfaces you want to use IPv6 with.

At this point, if you have IPv6-capable systems then you should be able to reset their network connections and get an IPv6 address assigned with the correct prefix. If you're starting out from a fairly vanilla OPNsense configuration then the DNS server address provided to the client should be the IPv6 address of the OPNsense router on whatever interface your device is connected to. You can test your IPv6 connectivity by going to https://test-ipv6.com/ or other similar sites.

The fourth task is stopping the router from providing DHCPv6 service on your internal networks, so that your own DHCPv6-capable server can do it, and also stopping the router from handing out its own address as a DNS server for clients to use, so clients will use your own DNS server(s) instead. This requires doing the following:

• Go to "Interfaces" > "LAN" (or OPT1, or whatever the name of the appropriate interface is).
• In the section "Track IPv6 Interface" check the checkbox "Allow manual adjustment of DHCPv6 and Router Advertisements".
• Click "Save".
• Go to "Services" > "Router Advertisements" > "LAN" (or "OPT1", or whatever).
• Change the value of "Router Advertisements" to "Managed". The router will then tell IPv6-capable clients that they need to talk to a DHCPv6 server to get an IPv6 address.
• Under "DNS options" check the box "Do not send any DNS configuration to clients". This prevents the router from sending a DNS server address in the router advertisements sent to IPv6-capable clients, so that the client will use only DNS addresses provided by the DHCPv6 server.
• Click "Save".
• Go to "Services" > "DHCPv6" > "LAN" (or "OPT1", or whatever).
• Uncheck the box "Enable DHCPv6 server on LAN [or OPT1, etc.] interface".
• Click "Save".
• Repeat the steps above for each interface on which you'll be deploying your own DHCPv6 and DNS servers.

The fifth task is making sure that the router is really not trying to provide DHCPv6 service. When I did the steps above and tried to set up my own DHCPv6 server, I noticed that IPv6-capable clients were somehow getting a bad IPv6 address, with a prefix that was no longer correct. In looking at a packet capture with Wireshark, I noticed that the clients were getting DHCPv6 replies from the router itself, telling the client to use the incorrect prefix. The client was using those replies rather than the ones from my own DHCPv6 server.

The problem (I discovered) is that the router keeps a record of past DHCPv6 leases that it issued, and apparently will send DHCPv6 replies when it sees a DHCP request from a client that it previously gave a lease to, even if the DHCPv6 service on the router is otherwise disabled. So, here's what you need to do next:

• Go to "Services" > "DHCPv6" > "Leases".
• If there are no leases displayed, then everything is OK. You can skip the remaining steps.
• If there are leases displayed, then you need to delete them. Unfortunately, there is no way to delete them in the GUI, so you will have to hack your way through this.
• Login to the router as root (via the console, or via SSH if you've enabled it).
• Type option "8" to enter the shell.
• Enter the command "cd /var/dhcpd/var/db" to go to the directory containing the DHCP server lease database.
• Enter the command "cat /dev/null >dhcpd6.leases" to wipe out the data for active DHCPv6 leases.
• Enter the command "cat /dev/null >dhcpd6.leases~" to wipe out the data for expired DHCPv6 leases.
• Enter the command "exit" to exit the root shell.
• Type option "6" to reboot the router. (Or in the GUI go to "Power" > "Reboot" and reboot it that way.)
• Once the router is rebooted, go to "Services" > "DHCPv6" > "Leases" and verify that there are no leases listed.

NOTE: You can and should do this for DHCPv4 as well, if you're going to run your own DHCPv4 server. The steps are the same as above, except that the DHCPv4 leases are in the files dhcpd.leases and dhcpd.leases~.

At this point you can configure your own DHCP and DNS servers on the interface subnets. How exactly to do this will vary between systems, but at a minimum you will need to do the following things for a DHCPv6 server:

• Configure the server to have a static IPv6 address using the currently-assigned /64 prefix for the subnet in which it is located (i.e., for the interface with which the subnet is associated).
• If the server can determine the IPv6 gateway automatically, configure it to do that. Otherwise configure it with the IPv6 address of the router on that subnet.
• Configure a DHCPv6 range on the subnet, with DHCPv6 options to provide a list of DNS server IPv6 addresses (option 23) and a list of domains to search (option 24).
• Optionally, configure the DHCPv6 server to do dynamic DNS updates to the DNS server.
• Configure the DHCPv6 server to listen to DHCPv6 requests originating from the subnet.

There are two other things I'm planning to do:

• Set up the third subnet to relay DHCPv4 and DHCPv6 requests to the DHCP servers on one of the first two subnets.
• Put firewall rules in place to disallow clients from sending IPv4 and IPv6 DNS requests to the OPNsense router and out the WAN interface. I'll maintain the OPNsense router as a DNS server, but only as a way for my own DNS servers to bootstrap themselves (e.g., to get NTP server addresses). (This is to mimic a typical customer deployment: since they will use their DNS servers to perform security checks on DNS queries -- e.g., blocking known malicious domains and doing behavioral analysis to detect data exfiltration over DNS -- they'll want to prevent clients, including malware, from bypassing the corporate DNS servers and going straight out to 8.8.8.8 or whatever.)

But that can wait for another time. In the meantime I am very happy with OPNsense (having recently migrated to it from pfSense), and wanted to show my gratitude by providing this tutorial, in case anyone else out there might find it of some use.

Frank

[pdhsker]

I just wanted to thank you for sharing your experience. Your post helped me set up IPv6 in my box.