OPNsense Forum

English Forums => 26.1, 26,4 Series => Topic started by: Ed V. on May 15, 2026, 07:26:09 PM

Title: Odd Kea DHCPv6 behavior...
Post by: Ed V. on May 15, 2026, 07:26:09 PM
So this one is weird.

If you recall, I'm the oddball that uses manual configs for my DHCP setup (the whole Cable Modem shared network /IP thing).

With the latest update to v26.1.8_5, the DHCP6 server has started exhibiting strange behavior.

Setting the stage...

The OpnSense Kea platform is the only DHCPv6 in my environment.

It appears to be the latest version from OpnSense:
# kea-dhcp6 -v
3.0.3

With the "Shared Networks" config in `kea-dhcp6.conf`, I can run the extended tests and they come back clean:

# kea-dhcp6 -T /usr/local/etc/kea/kea-dhcp6.conf
2026-05-15 12:10:01.791 WARN  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_MT_DISABLED_QUEUE_CONTROL disabling dhcp queue control when multi-threading is enabled.
2026-05-15 12:10:01.791 WARN  [kea-dhcp6.dhcp6/71095.0x4b07eb05c008] DHCP6_RESERVATIONS_LOOKUP_FIRST_ENABLED Multi-threading is enabled and host reservations lookup is always performed first.
2026-05-15 12:10:01.792 INFO  [kea-dhcp6.hooks/71095.0x4b07eb05c008] HOOKS_LIBRARY_CLOSED hooks library /usr/local/lib/kea/hooks/libdhcp_lease_cmds.so successfully closed
2026-05-15 12:10:01.793 INFO  [kea-dhcp6.hooks/71095.0x4b07eb05c008] HOOKS_LIBRARY_CLOSED hooks library /usr/local/lib/kea/hooks/libdhcp_host_cmds.so successfully closed
2026-05-15 12:10:01.793 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_NEW_SUBNET6 a new subnet has been added to configuration: 2001:579:4c:2700::/64 with params: t1=21600, t2=37800, preferred-lifetime=27000, valid-lifetime=43200, rapid-commit is false
2026-05-15 12:10:01.793 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_NEW_SUBNET6 a new subnet has been added to configuration: fde4:b3e2:db9e:1000::/64 with params: t1=21600, t2=37800, preferred-lifetime=27000, valid-lifetime=43200, rapid-commit is false
2026-05-15 12:10:01.794 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_NEW_SUBNET6 a new subnet has been added to configuration: 2001:579:4c:2701::/64 with params: t1=21600, t2=37800, preferred-lifetime=27000, valid-lifetime=43200
2026-05-15 12:10:01.794 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_NEW_SUBNET6 a new subnet has been added to configuration: fde4:b3e2:db9e:2000::/64 with params: t1=21600, t2=37800, preferred-lifetime=27000, valid-lifetime=43200
2026-05-15 12:10:01.794 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_SOCKET_TYPE_SELECT using socket type raw
2026-05-15 12:10:01.794 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_ADD_IFACE listening on interface ixl0
2026-05-15 12:10:01.794 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_CFGMGR_ADD_IFACE listening on interface vlan01
2026-05-15 12:10:01.796 INFO  [kea-dhcp6.lease-cmds-hooks/71095.0x4b07eb05c008] LEASE_CMDS_INIT_OK loading Lease Commands hooks library successful
2026-05-15 12:10:01.796 INFO  [kea-dhcp6.hooks/71095.0x4b07eb05c008] HOOKS_LIBRARY_LOADED hooks library /usr/local/lib/kea/hooks/libdhcp_lease_cmds.so successfully loaded
2026-05-15 12:10:01.797 INFO  [kea-dhcp6.host-cmds-hooks/71095.0x4b07eb05c008] HOST_CMDS_INIT_OK loading Host Commands hooks library successful
2026-05-15 12:10:01.797 INFO  [kea-dhcp6.hooks/71095.0x4b07eb05c008] HOOKS_LIBRARY_LOADED hooks library /usr/local/lib/kea/hooks/libdhcp_host_cmds.so successfully loaded
2026-05-15 12:10:01.797 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_MEMFILE_DB opening memory file lease database: persist=true type=memfile universe=6
2026-05-15 12:10:01.797 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /var/db/kea/kea-leases6.csv.2
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /var/db/kea/kea-leases6.csv
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_MEMFILE_BUILD_EXTENDED_INFO_TABLES6 building extended info tables saw 17 leases, extended info sanity checks modified 0 leases and 0 leases were entered into tables
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_MEMFILE_LFC_SETUP setting up the Lease File Cleanup interval to 3600 sec
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_LEASE_MGR_BACKENDS_REGISTERED the following lease backend types are available: memfile
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.hosts/71095.0x4b07eb05c008] HOSTS_BACKENDS_REGISTERED the following host backend types are available:
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.dhcpsrv/71095.0x4b07eb05c008] DHCPSRV_FORENSIC_BACKENDS_REGISTERED the following forensic backend types are available:
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.database/71095.0x4b07eb05c008] CONFIG_BACKENDS_REGISTERED the following config backend types are available:
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.host-cmds-hooks/71095.0x4b07eb05c008] HOST_CMDS_DEINIT_OK unloading Host Commands hooks library successful
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.lease-cmds-hooks/71095.0x4b07eb05c008] LEASE_CMDS_DEINIT_OK unloading Lease Commands hooks library successful
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.hooks/71095.0x4b07eb05c008] HOOKS_LIBRARY_CLOSED hooks library /usr/local/lib/kea/hooks/libdhcp_host_cmds.so successfully closed
2026-05-15 12:10:01.798 INFO  [kea-dhcp6.hooks/71095.0x4b07eb05c008] HOOKS_LIBRARY_CLOSED hooks library /usr/local/lib/kea/hooks/libdhcp_lease_cmds.so successfully closed

The server starts and tracks handing out addresses from the 2001:579:4c:2700:: range, but does not log anything in the fde4:b3e2:db9e:1000:: range.

2026-05-15T09:26:13-05:00Informationalkea-dhcp6INFO [kea-dhcp6.dhcpsrv.0x515279a5c008] DHCPSRV_MEMFILE_LFC_START starting Lease File Cleanup
2026-05-15T09:26:11-05:00Informationalkea-dhcp4INFO [kea-dhcp4.dhcpsrv.0x516d8645c008] DHCPSRV_MEMFILE_LFC_EXECUTE executing Lease File Cleanup using: /usr/local/sbin/kea-lfc -4 -x /var/db/kea/kea-leases4.csv.2 -i /var/db/kea/kea-leases4.csv.1 -o /var/db/kea/kea-leases4.csv.output -f /var/db/kea/kea-leases4.csv.completed -p /var/db/kea/kea-leases4.csv.pid -c ignored-path
2026-05-15T09:26:11-05:00Informationalkea-dhcp4INFO [kea-dhcp4.dhcpsrv.0x516d8645c008] DHCPSRV_MEMFILE_LFC_START starting Lease File Cleanup
2026-05-15T09:20:51-05:00Informationalkea-dhcp6INFO [kea-dhcp6.commands.0x515279a5c008] COMMAND_RECEIVED Received command 'lease6-get-all'
2026-05-15T09:20:51-05:00Informationalkea-dhcp6INFO [kea-dhcp6.commands.0x515279a5c008] COMMAND_RECEIVED Received command 'config-get'
2026-05-15T09:20:49-05:00Informationalkea-dhcp4INFO [kea-dhcp4.commands.0x516d8645c008] COMMAND_RECEIVED Received command 'lease4-get-all'
2026-05-15T09:20:49-05:00Informationalkea-dhcp4INFO [kea-dhcp4.commands.0x516d8645c008] COMMAND_RECEIVED Received command 'config-get'
2026-05-15T09:16:53-05:00Informationalkea-dhcp6INFO [kea-dhcp6.dhcp6.0x515279a76008] DHCP6_QUERY_LABEL received query: duid=[00:03:00:01:84:69:93:8f:d0:ca], [no hwaddr info], tid=0x6e1e19
2026-05-15T09:13:19-05:00Informationalkea-dhcp6INFO [kea-dhcp6.packets.0x515279a76008] DHCP6_PACKET_SEND duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4031d4: trying to send packet REPLY (type 7) from [ff02::1:2]:547 to [fe80::469:89b5:704e:d6e9]:546 on interface ixl0
2026-05-15T09:13:19-05:00Informationalkea-dhcp6INFO [kea-dhcp6.leases.0x515279a76008] DHCP6_LEASE_ALLOC duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4031d4: lease for address 2001:579:4c:2700::3 and iaid=0 has been allocated for 43200 seconds
2026-05-15T09:13:19-05:00Informationalkea-dhcp6INFO [kea-dhcp6.packets.0x515279a76008] DHCP6_PACKET_RECEIVED duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4031d4: REQUEST (type 3) received from fe80::469:89b5:704e:d6e9 to ff02::1:2 on interface ixl0
2026-05-15T09:13:19-05:00Informationalkea-dhcp6INFO [kea-dhcp6.dhcp6.0x515279a76008] DHCP6_QUERY_LABEL received query: duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4031d4
2026-05-15T09:13:18-05:00Informationalkea-dhcp6INFO [kea-dhcp6.packets.0x515279a76008] DHCP6_PACKET_SEND duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4acd6: trying to send packet ADVERTISE (type 2) from [ff02::1:2]:547 to [fe80::469:89b5:704e:d6e9]:546 on interface ixl0
2026-05-15T09:13:18-05:00Informationalkea-dhcp6INFO [kea-dhcp6.leases.0x515279a76008] DHCP6_LEASE_ADVERT duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4acd6: lease for address 2001:579:4c:2700::3 and iaid=0 will be advertised
2026-05-15T09:13:18-05:00Informationalkea-dhcp6INFO [kea-dhcp6.packets.0x515279a76008] DHCP6_PACKET_RECEIVED duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4acd6: SOLICIT (type 1) received from fe80::469:89b5:704e:d6e9 to ff02::1:2 on interface ixl0
2026-05-15T09:13:18-05:00Informationalkea-dhcp6INFO [kea-dhcp6.dhcp6.0x515279a76008] DHCP6_QUERY_LABEL received query: duid=[00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0], [no hwaddr info], tid=0x4acd6

Despite that, clients receive both 2001:579:4c:2700:: and fde4:b3e2:db9e:1000 addresses from OpnSense/Kea.

   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   IPv6 Address. . . . . . . . . . . : 2001:579:4c:2700::4(Preferred)
   Lease Obtained. . . . . . . . . . : Thursday, May 14, 2026 23:40:48
   Lease Expires . . . . . . . . . . : Friday, May 15, 2026 22:55:48
   IPv6 Address. . . . . . . . . . . : 2001:579:4c:2700:67e0:5d83:785d:2d2c(Preferred)
   IPv6 Address. . . . . . . . . . . : fde4:b3e2:db9e:1000:2941:55e:e973:29ad(Preferred)
   Temporary IPv6 Address. . . . . . : 2001:579:4c:2700:e992:445d:788e:8d84(Preferred)
   Temporary IPv6 Address. . . . . . : fde4:b3e2:db9e:1000:8141:11bc:3518:2088(Deprecated)
   Link-local IPv6 Address . . . . . : fe80::1f87:9cc:d92e:b807%11(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.144.21(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Lease Obtained. . . . . . . . . . : Tuesday, May 12, 2026 23:33:42
   Lease Expires . . . . . . . . . . : Saturday, May 16, 2026 04:22:55

On top of it all, even though there are leases recorded in the `/var/log/db/kea/kea-leases6.csv` and `/var/log/db/kea/kea-leases.csv.2` files, the WebUI reports no results found (screen capture attached).

# cat /var/db/kea/kea-leases6.csv.2
address,duid,valid_lifetime,expire,subnet_id,pref_lifetime,lease_type,iaid,prefix_len,fqdn_fwd,fqdn_rev,hostname,hwaddr,state,user_context,hwtype,hwaddr_source,pool_id
2001:579:4c:2700::1,00:03:00:01:ac:5a:f0:32:d4:46,43200,1778886008,1,27000,0,4029862982,128,1,1,dynamic-2001-579-4c-2700--1.lan.null-route.us.,ac:5a:f0:32:d4:46,0,,1,2,0
2001:579:4c:2700::2,00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0,43200,1778895409,1,27000,0,1,128,1,1,dynamic-2001-579-4c-2700--2.lan.null-route.us.,6c:7e:67:bb:73:f0,0,,1,2,0
2001:579:4c:2700::3,00:01:00:01:2d:a9:af:f9:6c:7e:67:bb:73:f0,43200,1778897599,1,27000,0,0,128,1,1,dynamic-2001-579-4c-2700--3.lan.null-route.us.,6c:7e:67:bb:73:f0,0,,1,2,0
2001:579:4c:2700::4,00:01:01:00:2f:06:d4:e6:9c:6b:00:aa:e2:94,43200,1778890248,1,27000,0,127691520,128,0,1,dynamic-2001-579-4c-2700--4.lan.null-route.us.,9c:6b:00:aa:e2:94,0,,256,2,0
2001:579:4c:2700::5,00:04:e1:18:49:99:32:fc:ad:1a:6e:cd:4b:f4:8c:94:e4:d2,43200,1778884562,1,27000,0,3055685611,128,1,1,dynamic-2001-579-4c-2700--5.lan.null-route.us.,,0,,,,0
2001:579:4c:2700::6,00:04:14:01:94:3d:5e:e1:43:c6:ef:f0:2f:6e:d3:15:9b:36,43200,1778889956,1,27000,0,726689589,128,1,1,dynamic-2001-579-4c-2700--6.lan.null-route.us.,,0,,,,0
2001:579:4c:2700::7,00:04:7d:69:f4:9b:51:6d:d2:61:44:7e:be:66:23:bb:f8:af,43200,1778890603,1,27000,0,4174257057,128,1,1,dynamic-2001-579-4c-2700--7.lan.null-route.us.,,0,,,,0
2001:579:4c:2700::8,00:04:47:f8:73:e6:1d:05:55:e8:aa:1b:a0:8a:7c:50:84:9b,43200,1778890230,1,27000,0,726689589,128,1,1,dynamic-2001-579-4c-2700--8.lan.null-route.us.,,0,,,,0
2001:579:4c:2700:aab:4e87:b033:ca45,00:03:00:01:bc:32:b2:a9:57:35,86392,1778896156,1,14392,0,0,128,1,1,,bc:32:b2:a9:57:35,4,,1,2,0
2001:579:4c:2700:1f60:865e:90da:b0cd,00:03:00:01:58:79:e0:21:af:00,86393,1778895689,1,14393,0,0,128,1,1,,58:79:e0:21:af:00,4,,1,2,0
2001:579:4c:2700:2177:9653:8512:3125,00:03:00:01:bc:32:b2:a9:57:35,86392,1778932177,1,14392,0,0,128,1,1,,bc:32:b2:a9:57:35,4,,1,2,0
2001:579:4c:2700:2659:ac73:43d1:1a7f,00:03:00:01:58:79:e0:21:af:00,86392,1778895688,1,14392,0,0,128,1,1,,58:79:e0:21:af:00,4,,1,2,0
2001:579:4c:2700:5df4:3144:fa64:a004,00:03:00:01:ac:c0:48:f0:79:89,86393,1778929395,1,14393,0,0,128,1,1,,ac:c0:48:f0:79:89,4,,1,2,0
2001:579:4c:2700:9efa:b659:3293:8bf2,00:03:00:01:bc:32:b2:a9:57:35,86393,1778896089,1,14393,0,0,128,1,1,,bc:32:b2:a9:57:35,4,,1,2,0
2001:579:4c:2700:c864:942c:b236:4510,00:03:00:01:ac:c0:48:f0:79:89,86393,1778929395,1,14393,0,0,128,1,1,,ac:c0:48:f0:79:89,4,,1,2,0
2001:579:4c:2700:cc20:ce97:a15c:252c,00:03:00:01:bc:32:b2:a9:57:35,86392,1778932178,1,14392,0,0,128,1,1,,bc:32:b2:a9:57:35,4,,1,2,0

My config files for Kea DHCP haven't changed since v25.* (and were working as expected in that release series), so I'm not sure what's going on here...

Any thoughts?  Clues?  Other things for me to dig out and post for review?
Title: Re: Odd Kea DHCPv6 behavior...
Post by: Monviech (Cedrik) on May 15, 2026, 08:04:57 PM
The GUI does not use the lease files anymore to display leases, it interacts directly with the unix socket now.

You need the leases hook library in your config file.

https://github.com/opnsense/core/blob/49b54ef032124e36eed2ad6fb19a9cc518f576a1/src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv6.php#L387

Afterward try executing the leases script to see if it returns anything:

https://github.com/opnsense/core/blob/master/src/opnsense/scripts/kea/get_kea_leases.py

Title: Re: Odd Kea DHCPv6 behavior...
Post by: Ed V. on May 15, 2026, 08:07:57 PM
The hook library is there (both dhcp6 and dhcp4):

    "hooks-libraries": [
      {
        "library": "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so"
      },
      {
        "library": "/usr/local/lib/kea/hooks/libdhcp_host_cmds.so"
      }
    ],

The script works - but only returns IPv4 leases...
Title: Re: Odd Kea DHCPv6 behavior...
Post by: Monviech (Cedrik) on May 15, 2026, 08:25:14 PM
Well tell the script to return IPv6 leases.

https://github.com/opnsense/core/blob/49b54ef032124e36eed2ad6fb19a9cc518f576a1/src/opnsense/scripts/kea/get_kea_leases.py#L106

--proto inet6
Title: Re: Odd Kea DHCPv6 behavior...
Post by: Ed V. on May 15, 2026, 08:59:45 PM
# /usr/local/opnsense/scripts/kea/get_kea_leases.py --proto inet6
#
# /usr/local/opnsense/scripts/kea/get_kea_leases.py --proto inet
{"records":[{"address":"192.168.144.3","prefix_len":128,"type":"","hwaddr":"24:5e:be:74:d2:4b","duid":"","client_id":"01:24:5e:be:74:d2:4b","iaid":"","valid_lifetime":86400,"expire":1778926767,"hostname"...

Title: Re: Odd Kea DHCPv6 behavior...
Post by: Monviech (Cedrik) on May 15, 2026, 09:25:23 PM
I just tried in my test installation on latest community and I get returns:

root@OPNsense:~ # /usr/local/opnsense/scripts/kea/get_kea_leases.py --proto inet6
{"records":[{"address":"fd10::1","prefix_len":128,"type":"IA_NA","hwaddr":"00:15:5d:00:ad:3e","duid":"00:01:00:01:31:3d:5f:3e:00:15:5d:00:ad:3c","client_id":"","iaid":3,"valid_lifetime":4000,"expire":1778876996,"hostname":"","state":0,"if":null,"if_descr":"","is_reserved":[]},{"address":"fd10:0:0:1000::","prefix_len":56,"type":"IA_PD","hwaddr":"00:15:5d:00:ad:3e","duid":"00:01:00:01:31:3d:5f:3e:00:15:5d:00:ad:3c","client_id":"","iaid":3,"valid_lifetime":4000,"expire":1778876996,"hostname":"","state":0,"if":null,"if_descr":"","is_reserved":[]}]}

So if nothing is returned, either KEA's socket really knows no leases, or there is some logic error somewhere (which I don't expect right now).

Quite strange.
Title: Re: Odd Kea DHCPv6 behavior...
Post by: Ed V. on May 15, 2026, 09:43:43 PM
Agreed - very strange.

I'm not a code wizard, but is there a way to directly monitor the hooks /sockets for activity?

Something relatively simple that I can use to help debug?

Or a way to set a flag for generating debug information in the hooks themselves?
Title: Re: Odd Kea DHCPv6 behavior...
Post by: Monviech (Cedrik) on May 15, 2026, 09:57:12 PM
Im not sure I can help debug this if you are not using the GUI to generate your current configuration.

We don't have a common baseline here.