dhclient exits on WAN IP renewal

[deviantintegral]

I'm running OPNsense 17.1.2 with a cable modem for the WAN, and I've run into issues when the modem does a fresh boot.

I imagine the way the modem (a Thomson DCM476) works is pretty common. On first boot, it issues a private IP, such as 192.168.100.1. That way, you can easily get to the status pages in case there's trouble getting modem sync. Once sync is detected, it passes DHCP through to the ISP. The initial lease has a 30 second expiry.

It's during this second phase that dhclient fails. Here's the logs from /tmp/igb1*:

==> igb1_output <==
dhclient: PREINIT
dhclient: Starting delete_old_states()
dhclient: Comparing IPs: Old: XXX.XXX.XXX.XXX New:
dhclient: Removing states from old IP 'XXX.XXX.XXX.XXX' (new IP '')
0 states cleared
killed 0 src nodes from 1 sources and 0 destinations
DHCPREQUEST on igb1 to 255.255.255.255 port 67
DHCPNAK from 192.168.100.1
DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 1
DHCPOFFER from 192.168.100.1
DHCPREQUEST on igb1 to 255.255.255.255 port 67
DHCPACK from 192.168.100.1

==> igb1_router <==
192.168.100.1

==> igb1_output <==
bound to 192.168.100.10 -- renewal in 30 seconds.

==> igb1_router <==
XXX.XXX.XXX.XXX

==> igb1_output <==
dhclient already running, pid: 83190.
exiting.

And in syslog:

opnsense: /usr/local/etc/rc.linkup: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf igb1 > /tmp/igb1_output 2> /tmp/igb1_error_output' returned exit code '1', the output was ''

The interface ends up marked as completely down.

If I manually renew the IP from the interfaces UI, or just run dhclient, everything works fine. Likewise, if I simply pull the ethernet cable, it renews correctly - it's only when the modem has lost sync and issues a private IP things fail.

Any suggestions?

[franco]

Hmmm, race condition during reconfigure... first dhclient is still there, second one can't execute. Let me take a look and report back.

[franco]

Hi there,

Possibly this, but I can't test right now and it is a pretty tough spot adding support for pid-based waiting for this one dhclient instance...

https://github.com/opnsense/core/commit/844d6b506

I don't recommend testing unless you know what to do / how to revert to a working state.


Cheers so far,
Franco

[franco]

Tested / revised patch for 17.1.x: https://github.com/opnsense/core/commit/bee13d3

# opnsense-patch bee13d3

(run again to remove patch)


Cheers,
Franco

[Mr.Goodcat]

Hi,

as I also get the "dhclient already running" in my logs, I applied the patch as well. Though it seems strange to me that only broadcasted DHCPRequests are answered, it might be the solution to my problems as well.

Here's the ouput the patch:

Code: [Select]

root@OPNSense:/tmp # opnsense-patch bee13d3
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From bee13d3f0fc4f88dc4f77495974ed67c0a90bcbb Mon Sep 17 00:00:00 2001
|From: Franco Fichtner <franco@opnsense.org>
|Date: Fri, 3 Mar 2017 20:47:06 +0100
|Subject: [PATCH] interfaces: better wait for dhclient exit
|
|PR: https://forum.opnsense.org/index.php?topic=4682.0
|
|(cherry picked from commit 844d6b506a56cfb525f0f45d893289bb41b3ba19)
|(cherry picked from commit b8c9eb4711c2e1b7edf43b231b4bb4716fbfd354)
|(cherry picked from commit 2d03d1966829035337aa30af985e72c36de8c088)
|---
| src/etc/inc/interfaces.inc | 10 +---------
| src/etc/inc/util.inc       | 18 +++++++++++-------
| 2 files changed, 12 insertions(+), 16 deletions(-)
|
|diff --git a/src/etc/inc/interfaces.inc b/src/etc/inc/interfaces.inc
|index 8c6d00d..6cd0d88 100644
|--- a/src/etc/inc/interfaces.inc
|+++ b/src/etc/inc/interfaces.inc
--------------------------
Patching file etc/inc/interfaces.inc using Plan A...
Hunk #1 succeeded at 2453.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/src/etc/inc/util.inc b/src/etc/inc/util.inc
|index a762b0b..e78d1fc 100644
|--- a/src/etc/inc/util.inc
|+++ b/src/etc/inc/util.inc
--------------------------
Patching file etc/inc/util.inc using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 41.
Hunk #3 succeeded at 60.
done
All patches have been applied successfully.  Have a nice day.


[franco]

Hey guys,

*awkward silence*

Any news on this?


Cheers,
Franco

[Mr.Goodcat]

Hi Franco,

sorry for the delay. I tried the patch and for a time it worked but then WAN<->LAN connectivity broke down again. However, due to the problem described in https://forum.opnsense.org/index.php?topic=4533.0 [1] I'm not sure which problem was the cause and my logs aren't really helpful. Thus I can't say with certainty if the patched fixed the problem described in this thread. At least it didn't seem to have a negative impact. Deviantintegral might have better feedback on this. Currently I'm testing with Pfsense (OPN's GUI is structurally so much better!) to see if the fault is with my hardware.

Off-topic [1]:
Also, I found this thread (in German) https://www.unitymediaforum.de/viewtopic.php?f=77&t=30742 which confirms my observation that the cable ISP's DHCP only answers to dhclient broadcasts. If this broadcasted rebind fails (e.g. due to lost UDP packets) or does not occur, a new WAN lease is obtained only after the lease expires and the whole process starts anew. However routing seems to fail as devices on my LAN bridge can ping each other but not out to WAN and OPN itself can't ping LAN devices but WAN works. Though the routing table looks fine to me, so I'm once again not sure what causes this. It would be great if one could force the WAN DHCP client to always send DHCPREQUEST to 255.255.255.255.

[deviantintegral]

Turns out I wasn't subscribed to this thread

Anyways, I applied the patch above, rebooted, and it didn't fix things for me. Same logs from dhclient.

Aside, nice utility with the opnsense-patch command. Running the same command again is a nice easy way to revert the patch.

[franco]

The patch hasn't made things worse so we'll add it in 17.1.4.

Now, instead, let's look at your file contents for:

/tmp/igb1_output
/tmp/igb1_error_output


Thanks,
Franco

[deviantintegral]

Here you go, running the patch linked to above. There's nothing logged in igb1_error_output.

==> igb1_output <==
dhclient: PREINIT
dhclient: Starting delete_old_states()
dhclient: Comparing IPs: Old: XXX.XXX.XXX.XXX New:
dhclient: Removing states from old IP 'XXX.XXX.XXX.XXX' (new IP '')
0 states cleared
killed 0 src nodes from 1 sources and 0 destinations
DHCPREQUEST on igb1 to 255.255.255.255 port 67
DHCPNAK from 192.168.100.1
DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
DHCPOFFER from 192.168.100.1
DHCPREQUEST on igb1 to 255.255.255.255 port 67
DHCPACK from 192.168.100.1
bound to 192.168.100.10 -- renewal in 30 seconds.
dhclient already running, pid: 92123.
exiting.


# Here's where I clicked renew in the web UI for the interface

dhclient: PREINIT
dhclient: Starting delete_old_states()
dhclient: Comparing IPs: Old: XXX.XXX.XXX.XXX New:
dhclient: Removing states from old IP 'XXX.XXX.XXX.XXX' (new IP '')
0 states cleared
killed 0 src nodes from 1 sources and 0 destinations
DHCPREQUEST on igb1 to 255.255.255.255 port 67
DHCPACK from XXX.XXX.XXX.XXX
bound to XXX.XXX.XXX.XXX -- renewal in 55164 seconds.

[deviantintegral]

It looks like this is still an issue on 17.1.6. Is there an upstream issue I can track?

[franco]

Hi there,

There isn't, but we're reworking all IP renewal at the moment due to numerous pushes and questions. Best thread to track is this:

https://forum.opnsense.org/index.php?topic=4900.0

Latest EXPERIMENTAL patch (third iteration) applies on top of 17.1.6:

https://github.com/opnsense/core/commit/3b315ad

To install:

# opnsense-patch 3b315ad

To revert run the command again (but I must say this requires internet connectivity which could fail).


Cheers,
Franco