Config not saving to disk

[np19]

OPNsense 21.1-amd64
FreeBSD 12.1-RELEASE-p12-HBSD
OpenSSL 1.1.1i 8 Dec 2020

We have had an issue with the config saving to disk.  If we make changes in the web GUI and then the firewall hard restarts for any reason (power loss, etc) even if it is several hours after the change, it will revert all those changes as if they never existed (they are not even in the config backup list).  This does not appear to affect changes made in the console (interface assignment, IP addressing, etc).  Restarting the firewall after making a change appears to save the changes against a hard restart.

We are running firewalls in a Cloudstack cluster, using KVM and nfs storage.  We first noticed this when a host failed and the running firewalls reverted to their initial config (interface assignment, etc stayed, so it wasn't an issue with the disks themselves, but the config coming from the web GUI.)

Has anyone experienced this and is there any way to ensure all changes are immediately flushed to disk?

[Sheldon]

[...] it will revert all those changes as if they never existed (they are not even in the config backup list).

I assume the "config backup list" is the one available via System -> Configuration -> History.

When you make any configuration changes, are these changes listed in that history in the first place, before restart?

[np19]

Thank you for the reply.  Yes, this was the location I meant, but I have not checked whether it shows them there before the restart.  I can do some testing around that.