Messages

I ran across an interesting problem today as I was trying to figure out why my opnsense firewall had died. It went unresponsive and after rebooting it I found the root filesystem full. As I investigated I determined that there were log files in /var/log/dhcpd that were 5-7GB in size for my little home network with less than 40 devices. I removed the largest files from the last few days and things started working better.

I started tailing /var/log/dhcpd/latest.log to see what was being written and it was writing the same thing over and over so fast that the sequenceId value just looked like a counter in an infinite loop showing the same 4 lines over and over.

<190>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184686"] Renew message from fe80::42b0:34ff:fe9f:47de port 546, transaction ID 0x250FF600
<190>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184687"] Reply NA: address 2602:3f:e615:<redacted> to client with duid 00:03:00:01:40:b0:34:9f:47:de iaid = 2 valid for 7200 seconds
<191>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184688"] Reusing lease for: 2602:3f:e615:<redacted>, age 650 secs < 25%, sending shortened lifetimes - preferred: 1, valid 6550
<190>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184689"] Sending Reply to fe80::42b0:34ff:fe9f:47de port 546
<190>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184690"] Renew message from fe80::42b0:34ff:fe9f:47de port 546, transaction ID 0x2C7EB000
<190>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184691"] Reply NA: address 2602:3f:e615:<redacted> to client with duid 00:03:00:01:40:b0:34:9f:47:de iaid = 2 valid for 7200 seconds
<191>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184692"] Reusing lease for: 2602:3f:e615:<redacted>, age 650 secs < 25%, sending shortened lifetimes - preferred: 1, valid 6550
<190>1 2024-06-16T08:39:12-06:00 OPNsense dhcpd 35452 - [meta sequenceId="184693"] Sending Reply to fe80::42b0:34ff:fe9f:47de port 546

It didn't take long to figure out that the link local address was for my LaserJet Pro MFP M277c6 which doesn't have any IPv6 configuration options on its UI.

As I was trying to figure out how to just ignore this device I decided to try assigning it a static DHCP6 reservation. After configuring a reservation for it with an IPv6 address and a description it was able to take its address and start working normally without churning through the same 4 renewal steps over and over.

I'm guessing this was a problem caused by a badly behaving printer, but it was a strange enough problem I wanted to make sure what had happened and what seems to have resolved it got recorded somewhere in case someone else runs into a similar issue.

Versions:
opnsense 24.1.8
isc-dhcp44-server 4.4.3P1_1
Not using kea dhcp server

I haven't been on opnsense 24 for long, I upgraded from 23 series in the last 2 weeks.

I'm using the Zyxel box because it makes my life easier if I have to contact Centurylink support. Having their box terminate the final handoff of the fiber connection means that it is the hardware they supplied and provisioned that is doing the PPPoE authentication. As a matter of convenience and ease of support I want that device owning the gateway for my public subnet so that everything else can simply be a static IP configuration in IPv4 or IPv6.

I have found that trying to replace the service provider's modem instead of letting it be the PPPoE terminator and public address space gateway just leads to headaches. Many years ago I tried to help someone with their pfSense that was trying to own the PPPoE instead of letting the provider's equipment do it and there was nothing but suffering and finger pointing about whose equipment was doing it wrong.

[The C3000Z Configuration:]

I have been fighting with my IPv6 configuration on my OpnSense box. It would work when I reconfigured the interface, but about 30 minutes later IPv6 would break. I wanted to write up my observations so that hopefully someone in a similar situation can have a useful search result when trying to figure out why their IPv6 is breaking after their router advertisements expire.

The C3000Z Configuration:
I get a /29 IPv4 subnet from my provider and a /56 IPv6 subnet delegation. My modem runs ethernet to my fiber provider's ONU on the wan side, handles authentication, and presents the /29 on the LAN side. Out of the /56 delegation I define one /64 to live on the same subnet as my public IPv4 /29. The LAN network on my modem is the same network segment that is on the WAN side of my OPNsense box.

There are some limitations on what I can control on the C3000Z so I'm kind of at the mercy of how I can get it to function. Between DHCPv6 and stateless I configured it in stateless mode with a 30 minutes RA expiration. I selected a static IP in the /64 network and created a static route entry pointing a /60 prefix to it.

The OPNsense Configuration:
On my OPNsense box I setup my WAN interface IPv6 in static IP mode using the address I defined as the next hop target for the /60 subnet on the C3000Z. I also take one of the /64 networks from the /60 delegation and configure that with DHCPv6 on the LAN interface.

Notably when I configured the IPv6 on the WAN interface I was having trouble getting the modem to tell me what its IPv6 address was on that interface so I just saved and activated the configuration with the "IPv6 Upstream Gateway" set to "Auto-detect" and my IPv6 started working.

It all seemed to work, but there was a catch

I thought at this point that I had it all working. My computers were getting IPv6 addresses and everything was working. However, the next morning I wasn't able to connect to IPv6 hosts anymore. In my continued testing every time I would touch the configuration and apply it to the interface the IPv6 would start working, but I started to notice that it stopped about 30 minutes later. I figured this was because it was getting a router advertisement and then forgetting it. I couldn't work out why it wasn't just continuing to respect the RA it was getting. When it was working I saw a default route in the output in System -> Routes -> Status, but the default would eventually disappear from the list of ipv6 routes.

As I was trying to diagnose what the trouble was and lamenting that I couldn't just set a static IP on the C3000Z to use as a static IP for my gateway I looked at the routing table again and noticed that the default route was just pointing to the fe80::/64 local network autoconfig address that is based on my modem's mac address.

The thing I had been worried about was the possibility that the upstream address in the router advertisement could change and that the gateway address could also change as a result, but I stopped worrying when I saw that it was the local address based on the mac. Once I realized this I created an upstream gateway and set my router's fe80:: autoconfig address as the next hop and my IPv6 has continued to work even after the normal RA expiration time when it was breaking before.

What I suspect was the problem
I think it is just a little bit of luck that made it work in the first place. My modem gives me precious little information and I struggled to identify if it was even binding any kind of an address in the /64 IPv6 subnet I had defined. I suspect that when OPNsense was configuring my WAN interface it respected a router advertisement it received while configuring the interface, but when it eventually expired no additional advertisements were accepted because the interface was in static IPv6 configuration mode. By finding a reasonable next hop address for my modem and configuring that as the IPv6 Upstream Gateway on my WAN interface I believe that it is just asserting a correct gateway route instead of temporarily getting a next hop and forgetting once it expires.