WAN not pulling IP on power restore

[rkubes]

I have opnSense running on a mini PC and there is a ONT that serves as my modem. They are both protected by the same UPS.

Today I've found that if the UPS loses power and both need to boot up at the same time, opnSense finishes booting before the ONT. It detects the Ethernet link before the ONT boots; but of course can't pull an IP so it just lists the WAN interface as down.

Once the ONT finishes booting, it doesn't seem like opnSense retries pulling an IP, so the WAN stays down until I manually log in and tell opnSense to renew the lease on the WAN interface.

If I only reboot the router without rebooting the ONT, everything works as expected and an IP is pulled at boot.

Is there a way to get opnSense to retry to account for the ONT booting up? Or any other suggested solution so that it doesn't just give up on WAN and require manual intervention?

[rkubes]

I tried again yesterday while doing a different test. This time it did pull an IP at first, but shortly after it had packet loss and dropped the IP, which is understandable, and then eventually it tried again and after another minute it pulled an IP again and started working.

So, it seems like if it pulls an IP while booting, the retry logic works fine.

However, if it doesn't pull the IP while booting, it doesn't seem to retry unless you manually tell it to renew the lease.

I'm using the default settings, which I believe is set to retry every minute for 15 minutes, if that's helpful.

[rkubes]

I found the offending log entry, but I don't know how to fix it.

Basically, dhclient complains there was no link and says it's giving up. However, once igc0 (WAN) link is up, there is nothing that triggers dhclient to run/try again. So, it just stays down until I manually trigger it from the UI (Interfaces/Overview page).


For more details:

In the system log, as it's booting up, at 15:53:51 you can see it starts to discover the igc0 NIC, and will list its MAC address
15:53:56 (5 seconds later) it says "<118>Configuring WAN interface"
15:54:04 (8 seconds later) it says "/usr/local/etc/rc.bootup: The command '/sbin/dhclient -c '/var/etc/dhclient_wan.conf' -p '/var/run/dhclient.igc0.pid' 'igc0'' returned exit code '1', the output was 'igc0: no link .............. giving up'"
15:54:18 (14 seconds later) it says "<6>igc0: link state changed to UP"
15:54:52 (34 seconds later) it says "<118> WAN (igc0) ->"

On that last log entry, it would normally list the WAN IP address, but it never pulled one because dhclient gave up, and nothing try to get dhclient to go again after igc0 link reestablished.


Is this expected behavior? Is there any way I can configure it so that dhclient does attempt again once igc0 is up? Or should I just script an "every minute" cron task to check if dhclient for WAN is running and restart it if it isn't?

I'm hoping for a clean way to resolve this rather than putting in a hack.


Edit:
I think I'm getting very close to a solution. What I've found is the "/usr/local/etc/rc.linkup" is what is supposed to trigger. However, it has a statement early in the script that prevents it from triggering during bootup. There's likely a reason for this to ensure it doesn't trigger "too early" - however, because it doesn't trigger during the bootup, there is also nothing to trigger again later.

It ultimately is a race condition where if the interface is down during boot, when the dhclient first runs, but the interface comes up before the boot process finishes, then the interface will not be fully configured.

I think to avoid modifying any core files (since I don't plan to submit a full patch for this), I'm just going to create a script to just see if dhclient is not running for wan/igc0, and the boot process has finished, then trigger the rc.linkup script for igc0.

[sanji]

This is very interesting. I'm having similar issues after power losses/outages.

What I always did was diconnect the modem from the Opnsense box, reboot the modem, reboot Opnsense, wait until both are fully booted and the modem has internet connectivity, then connect them both again. A real hurdle but I didn't find any other solution.

What surprises me is that all these threads always seem to be ignored. If you search for Opnsense and power outage or something similar, there are a lot of threads/posts etc. since years.