chrony stopped WebUI

[thogru]

Hi everyone,

I want to test the chrony plugin.

My current version info is:
OPNsense 21.1.6-amd64
FreeBSD 12.1-RELEASE-p16-HBSD
OpenSSL 1.1.1k 25 Mar 2021

I freshly installed 21.1.5 an did an update to 21.1.6. The onyl other plugin installed is os-dyndns (uninstalled it first to get ridd of the "misconfigured" message).

I currently use ntp and the associated daemon.

I install the chrony plugin (version 1.3).

Everytime I looked in the configuration of the chrony plugin the WebUI did not responed any more to my clicks in the browser (Firefox 89.0 (64-Bit)). The left pane (showing the menu) still respones to clicks (I can fold unfold the menu) but I can not enter any pane on the right side. When I click on Lobby -> Dashboard the browser waits "for ever" and does not update the right pane. The right pane still shows the content of the Services -> Chrony -> General pane.

The only sulotion to get out of this "lockin" is to navigate with the browser to an other pane (e.g. Lobby -> Dashboard), clear the browser cache and reboot OPNsense (via ssh or serial interface). After this procedure you can log in to the WebUI again and the WebUI will work again.

While the WebUI is locked everything else in OPNsense seems to work: ssh connection, serial communication, the firewall seems to work, the configured routing between interfaces still works, ...

Output of sudo top:

Code Select Expand


last pid:   137;  load averages:  0.38,  0.26,  0.12                        up 0+00:04:40  19:25:30
35 processes:  1 running, 34 sleeping
CPU:  0.2% user,  0.0% nice,  0.2% system,  0.1% interrupt, 99.5% idle
Mem: 127M Active, 19M Inact, 207M Wired, 99M Buf, 1478M Free
Swap:

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
80775 root          1  20    0  1038M  3944K CPU0     0   0:00   0.25% top
43593 root          1  20    0    27M    17M select   2   0:02   0.07% python3.7
54470 root          1  20    0    20M    10M select   0   0:01   0.05% python3.7
60270 root          1  20    0    21M    11M select   0   0:01   0.04% python3.7
31950 root          1  20    0  1036M  2868K select   2   0:00   0.04% powerd
47833 root          1  20    0  1050M  7108K select   3   0:00   0.02% ntpd

There is no load on the OPNsense. I can not see a process causing the lock of the WebUI.

Is there a way to get the chrony plugin running on fresh installations?

Can you recreate this issue on your developing machines? Do you need further information about the issue? As there are others reporting the same issue (like https://forum.opnsense.org/index.php?topic=23047.0) I think this issue is not related to my individual configuration.

If you want me to open a ticket in your ticket tracking system please let me know how to do this.

Kind Regards,
Thomas

[gpb]

Just tested and cannot recreate per your setup (firefox, 1.3, 21.1.6).  I don't use dyndns.  Also, when I enabled chrony, I made sure to disable the default ntp service by removing the interfaces.  I also changed the port for chrony to 123.  And now using NTS cloudflare...but I doubt that matters.  I can click through all the tabs and continue on without issue.

[psychofaktory]

Hi,
I've the same behaviour like thogru.

As soon as I try to save the chrony-settings, the WebGui is not responding anymore.

The default NTP was disabled as from gpb described an the port for chrony changed to 123.
I testet to leave the default port of chrony untouched, but the WebUI didn't resonse also.

I use DynDNS (if that matters) and I did not use NTS.

[thogru]

Hi everyone,

I uninstalled the dyndns plugin for this test to avoid interfearance.

Perhaps I did not express it well enough. The problem is not that the time synchronization fails when using chrony nor that some setting of chrony do not work correctly.

The matter is, that the WebUI simply do not respond to any clicks outside the chrony "pane".

Please find attached a short video give you an visual impression:

• I viewed the dashboard (about 3 seconds)
• I navigate to the chrony pane in the menu (until 6.166)
• I change to several pane within the chrony general pane (until 8.916)
• I clicked on the OPNsense-Logo to switch to the dashboard (round about 10.0)
• after this the WebUI hang, the activity indication of Firefox seems to show activities, the dashboard will never appear

Hopefully this description explains the issue better.

During my tests I recognized that this issue does not appear evertime when I want "to leave" the chrony plugin. I would assume about a 50 percent chances not to be able to leave the chrony panes and need to reboot OPNsense.

My OPNsense is running on the small APU (specifications here https://pcengines.ch/apu4d2.htm).

Please let me know if I can provide more information.

Kind Regards,
Thomas

[gpb]

Retested and still cannot reproduce.  Do you have any strange add-ons in the browser?  Maybe try chrome (or other) or use a mobile device to verify it's opnsense.  Not sure if there would be anything in the logs...in case you hadn't checked.

[thogru]

Hi gpb,

Thanks for testing again.

I installed the chrome browser in a virtual machine running windows 10. I was able to reproduce the issue. Furthermore I was not able to connect to my OPNsense from an other pc.

BUT after a long time (I would guess more than 10 minutes, I did not measure because some shopping took my focus) I can connect to the WebUI again. Based on this discovery I did further tests.

I established three connections to the WebUI of my OPNsense:

• from chrome browser in my VM
• from Firefox browser in my VM
• from Firefox browser on my real PC

I used the following procedure to do some measurements:

• Navigate to some pane for within each browser
• in chrome click though the chrony plugin, test if dashboard is reachable
• if dashboard is answering, continue with step 2
• take the time until browsers display correct WebUI pane

Please find attached the logs. Here are the timestamps which are relevant for the tests:

• at 19:00 I restarted OPNsense to get a working test environment
• around 20:40 all three browser hung for 30 up to 60 seconds
• at 20:47 all browser hung for about 4.5 up to 5 minutes
• from 20:56 the 3rd connection hung for about 60 seconds, the 2nd connection hung for 90 seconds and the 1st connection hung for 140 seconds
• after  these tests I rebooted OPNsense to get rid of the delays during navigation.

Please find attached the logs from system.

Kind Regards
Thomas

[gpb]

Since you went through this testing, I would c/p it to github as a bug...just follow the template and hopefully it will get some attention.

https://github.com/opnsense/plugins/issues

[giangi]

I was also trying out chrony just today. Same UI freeze. ntpd stopped and completely restart. The gui becomes unresponsive and either becomes working again with a complete restart (option 11 from console does not do the trick) or after waiting a long enough time to forget about it.

Running chronyc sources/tracking gives a "506 Cannot talk to daemon", which from a quick google search sounds like it is not running at all.
If there's anything I can do to be of possible help, lmk

Edit: well, I restarted again to unfreeze the gui (bc of my stubbornness in pressing save), and surprise, this time it started fine...

[gpb]

You disabled NTP?  As I remember when I removed the interfaces (Services | Network Time) it was servicing I also had to stop the service from the dashboard page.

[mimugmail]

So it seems the Diagnostics commands dont have sane timeouts, and when the daemon isnt started there start to many of them hanging around