HA - Network Time using VIP on backup node

[tt-ah]

Hi everyone!

I have a HA-Cluster with 2 nodes running on 23.1.11.

I am trying to upgrade but I see 2 issues causing me to hesitate:

• NTP not syncing anymore on backup node
• Weird failover after config-sync (see: https://forum.opnsense.org/index.php?topic=34140.msg181801#msg181801

I noticed time being out of sync on the backup node and network time>Status showing Unreach/Pending for all configured NTP servers.
While troubleshooting this I noticed that the network time service on the backup node uses the VIP of the WAN-interface to send its requests to the ntp servers instead of its own address on WAN. This struck me as stupid, since the replies will go to the master node.
No other service on the backup behaves like this to my knowledge. DNS and other outgoing connections like backup to gitlab/Nextcloud work fine.

If I SSH into the host and use ntpdate it will use its own IP and thus be able to sync time. When I stop the network time service and use ntpdate *ntp-server* I can successfully set the current time.

CARP is running on WAN and on several VLANs on LAN. Additionally there is an alias-IP on WAN used to SNAT some local subnets. I have state-sync and config-sync for network time (and more) configured. Config sync is disabled for static routes, but none are configured.

I can not tell exactly when this started, I just know with certainty that it has not been an issue up until I started upgrading the cluster step-by-step from 21.x onwards.

So far I have tried

• restarting the network time service
• rebooting the node
• outbound SNAT rule to SNAT traffic to ntp-server to its own IP
• searching for similar issues

Can someone tell me what I could look for? It feels like a bug to me but I see no reports of this behaviour from anyone else.

[opnsenuser]

Hi,
does the backup node have a route and is allowed to a ntp source that is configured in Services > Network Time > General?

I'd use an internal interface at which both firewalls have an ip & ntp is allowed and configure the other firewall as a timesource for the local system.
This way you can be sure the times don't get out of sync, even if there are shenanigans with carp.

BR
 

[tt-ah]

Sorry for the late response, I did not get an e-mail notification.

The backup node has a default route which makes the upstream ntp-servers available.
The only issue is, that the ntp-service uses the CARP_VIP of the WAN-Interface when trying to reach those ntp-servers.

I am not sure, if I can setup the master-node as NTP-Server for the backup-node as a workaround, because the ntp-service is replicated using Config-Sync. This means the master-node would also use itself as ntp-server. Could this cause problems during normal operation?

I have another 2-node CARP-Setup (which is our internal ntp-server) where his exact setup works as intended. Here the backup node uses its own IP von the WAN-Interface, instead of the CARP_VIP to communicate with the upstream servers.