NTP not working with Wifi Mesh network devices assigned dynamic addresses

[RobLatour]

I don't know if this is an OPNSense bug or not - so I thought I would post here and get comments before reporting as a bug (if appropriate) on Github.


In short: NTP is not working for devices on my Wifi Mesh network that are assigned dynamic addresses, however it does work if I assign them a static address.

Question: is this an OPNSense bug or just the way things should work?


Additional background and detail:

I am running OPNSense (current community version) with various interfaces, including:
   LAN
   LAN_IOT
   Master_Clock

I have two separate physical Wi-Fi networks.  One is attached to LAN the other to LAN_IOT.
Both the Wi-Fi LAN and LAN_IOT are running in Access Point mode.
The Wi-Fi LAN device is a TP-Link Deco mesh system (with three TP-Link Deco connected in the Mesh). 
The Wi-Fi LAN_IOT is a legacy TP-Link Access Point with also connects to a legacy TP-Line Extender.

OPNSense is running the Network Time Service.

The Network Time Service runs on the Master_Clock interface.

The Network Time Service gets its time from an ESP32 - GPS based Stratum 1 Time Server I developed and released in 2023.

The Network Time Service delivers NTP date/time data to all other interfaces.

This setup has been working very well for me since 2023, however almost all my devices connected on all interfaces have been set up, via OPNSense, with static IP addresses.

Recently I have been programming another set of ESP32 projects and noticed that while there could connect to either the LAN Wi-Fi network or the LAN_IOT Wi-Fi network they would not retrieve the NTP date/time data.

I spent several days trying to resolve this; but most of this time was chasing down the rabbit hole that my ESP32 code was bad / bad board configurations / etc (non of which seemed to be the issue in the end).

Also, in debugging this, at one point I stopped the OPNSense Time Service, and with it stopped my other device could get NTP date/time data but my ESP32 projects could not.  So that was not it.

However, late yesterday I found that if in OPNSense I set the ESP32 project device up with a static IP address it was able to get the NTP date/time date just fine. However when it ran as with a dynamic IP address the problem returned.

In conclusion:

In any case, I am reporting it here. 

I don't know if its an issue with OPNsense, TP-Link, both. or simply how things work.

If it is an as of yet unknown/unreported issue with OPNSense, the above offers a work around.

Also, if the comments here so indicate, I will open it up as a bug on the OPNSense Github page.

With thanks.

[meyergru]

DHCP is able to announce NTP services, so if a difference exists between static and dynamic configurations, it is highly likely that the problem lies there.

How do you configure your NTP client in each setup? If it does not work with DHCP, then I can imagine two problems:

1. You want to get the NTP server via DHCP and you do not get it - this can be because the DHCP/BOOTP packet length is limited and depending on what other info you may be packing into DHCP, some platforms may ignore less vital informations. Whe had a case similar to this.

2. You want to set NTP server statically and DHCP gets in the way by setting it. Maybe, it gets set to 0.0.0.0 by the DHCP client.

Either way, I would assume a problem in the client configuration or software or DHCP limit overrun, mostly because I know that NTP works even when devices are using DHCP.

You can easily check with another client and/or by dumping DHCP requests and answers.

[RobLatour]

Thanks for your comments.

How do you configure your NTP client in each setup?

There are different ways to get NTP data for an esp32 device - but under the hood I suspect they all boil down to the same thing:
Here is some stub code I wrote for an emulator to see if I had the basics right (which I did):
https://wokwi.com/projects/420011361310192641

You can easily check with another client and/or by dumping DHCP requests and answers.
In my testing I set up Wireshark on another computer on the same interface, and the had it monitor for ntp requests.
Yesterday, When the esp32 device had a dynamically assigned address I could not see the ntp request via wireshark.
Also, yesterday, when the esp32 device had a statically assigned address I thought could see the ntp request via wireshark, but I just checked again and I didn't see it.

Not using Wireshark, is there a way in OPNSense to dump DHCP requests and answers?

[meyergru]

Of course. Wireshark is just a fancy GUI for packet dumps. You can just use tcpdump on OpnSense CLI itself.

BTW: I do not doubt that there are no NTP packets. The big question is why there are none when clients are configured via DHCP.
To answer my question: Obviously, you set the NTP servers in your code, so your client does not use DHCP assignments. So the only thing I can imagine is that when you do static configuration, you give a DNS server, whereas the DNS server is not contained in your DHCP (did you set it in OpnSense?) or your client does only honor IP and routing information from DHCP.

[RobLatour]

Hi,

Thanks for your comments.

Regarding: Obviously, you set the NTP servers in your code, so your client does not use DHCP assignments.

That's not exactly how it works, if you look at the example from my earlier post here: https://wokwi.com/projects/420011361310192641 you will notice that all that is done is a wifi connection.  The NTP request, travels over UDP, and I assume is generally broadcast and that is how the NTP sever pick up on the request.  I also assume that the NTP server gets the source IP address from the UPD packet and that is how it know to which device it needs to respond to (if it itself also doesn't do a general broadcast). In any case I am no expert on how NTP Servers work, but that is my educated guess as the client doesn't configure the NTP server's address.

Again, this works fine when the client is connected with a static address, but with a dynamic one.

[meyergru]

You explicitely call

Code Select Expand

configTzTime("EST5EDT,M3.2.0,M11.1.0", ntpServer1, ntpServer2);, where both variables are explicitely set in your client. You do not have an OS mechanism where the NTP server is being extracted from a DHCP response. So, either way, the NTP servers are configured statically.

Thus, if there are no NTP connections tried with DHCP over static network configuration, I would assume that the DNS resolution of the explicit names "time.nist.gov" and "pool.ntp.org" is the culprit.

[RobLatour]

Well of course you are right! :-)

I had been thinking of my ntp server as the Network Time Service running on OPNSense - i.e. the ip address of the OPNSense box itself - which as I understand it intercepts calls to, for example, pool.ntp.org, and responds to the device itself.

[meyergru]

It will, but only if you ask it to. WIth NTP-capable DHCP clients, you can do this via DHCP, otherwise you will have to specify your OpnSense box explicitely as the server.

[RobLatour]

Well, I change my project's code to reference only 1 NPT Server (as shown below)

configTime(GMT_OFFSET_SEC, DAY_LIGHT_OFFSET_SEC, NTP_SERVER1);

and point directly to the ip address of my esp32 time server used by OPNSense.

However, I got the same results:

If the client is assigned a dynamic IP address it does not get the time from the time server.

If the client is assigned a status IP address it does get the time from the time server.

[meyergru]

This is beyond me. It must be something in the low-level configuration. DHCP announces IP addresses, the default route, the DNS server and - potentially - the NTP server. The last one can be ruled out, since you give that in your application.

If you are right, then DNS can be ruled out as well, since you give the IP address.

So, if there is IP connectivity at all, it should be independent on how you configure your machines. On the other hand, who knows? But even if the default gateway is misconfigured or not honored, the machines should see one another if they are on the same subnet.

I would look at everything low-level now, like: netmasks, gateways, routes. There must be something different with static/DHCP setups.

[RobLatour]

Thank you for your time and insights.  This is a well over my paid grade, so I really do appreciate your help.  I'll poke around a little more on this and may open it on github as potential bug.  However, for now I've got my program up and running, just had to assign a static ip address to the device running it. 

Thing is it took several days until I stumbled on this work work around.  However, perhaps someone else reading this thread in the future will be able to save some time because of it. 

Again, with thanks for your time and insights!

[Patrick M. Hausen]

I place a small bet on netmask issue 🙂