OPNsense Forum
Archive => 18.7 Legacy Series => Topic started by: drivera on October 28, 2018, 05:19:46 am
-
Hi!
I found a GitHub issue report (https://github.com/opnsense/plugins/pull/458) which suggests that automatic, periodic configuration backup via SCP has already been implemented and merged into master since January, but I've found nothing relating to it within the current release's UI (18.7.6).
Also, the plugin itself (scp-backup) seems to have disappeared, further suggesting that this got added to core as the above link suggests.
However, I can't find the configuration page anywhere. Did this get scrapped? Did it get pushed out to the 19.X release tree?
This is important functionality for those of us who need to keep backups locally (due to company policies, for instance). I'd settle for a backup via a network mount (NFS? Samba isn't ideal...but better than nothing)...
I can code this manually, but I'd just as soon have a UI to administer it, for consistency's sake (also, it's easier to backup a single config XML that has everything, including its "self-backup" configuration).
Thanks!
-
When you have master installed you can see this section on the backup page where also gmail and nextcloud sits.
But I'm quite sure it's not working .. just test it
-
I'm pretty sure I have master installed - rather: I installed from the USB drive image (vga) - is this not based on master? Further, I ran updates to 18.7.6 ... would that not include that functionality?
Am I misunderstanding how OPNSense releases are built/structured? (if so, can you point me to where I might get up to speed? :D)
Thanks for the quick replies!
-
No, you dont. "master" is the current development branch. There are also branches for 18.7 and so on .. when someone sends a fix it lands in master. When it's verified to work it'll be merged to 18.7 branch and get in the latest release.
On a test machine (and only on a test machine) go to CLI:
opnsense-update -t opnsense-devel
opnsense-code core
cd /usr/core && make upgrade
Then you are on master :)
-
Ok... I guess what I meant to ask was "this was merged in January - shouldn't it have made the stable/prod branch by now?" :D
But you're 100% correct.
So I guess this feature never made it to the 18.7 branch (since 18.7.6 only came a few days ago)?
Thanks!
-
Yep, I'm quite sure there's something missing which prevents adding it to stable.
-
FYI. It is there but it currently has no implementation in the class is empty, so it is not operational:
https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/library/OPNsense/Backup/Scp.php#L115
The core class is what Franco wanted to move over from the plugins repository but did never finish it so it is still a stub and also won't run on alpha / beta even if visible there.
The plugin should be functional but it is not intended for release (should be moved to core).
-
I can probably take a look at that as well.
-
Another interesting thought: backup to remote GIT (SVN?)
Still has security implications with all the unencrypted stuff that goes in that XML, which kinda begs the question: doesn't it work better to just encrypt the "sensitive" stuff vs. (optionally?) the whole thing? It would make life simpler with regards to change tracking with something like GIT/SVN as a backup mechanism.
Thoughts?
-
GIT may produce a disk space issue and encrypted is against the principle of git (maintain the diffs).
Also I would need something like https://github.com/libgit2/rugged to write such an adapter because I would have to access the repository programmatically.
-
Well, since the intent would be to back up to an external GIT repository, and it'd only be config.xml, and I'm fairly sure GIT has the means to reduce local space consumption by trimming unnecessary history, this avenue might actually turn out to not be a bad idea.
I'll try to think up an approach and see if I can come up with a coherent proposal for it. I see value there, but you're right: the encryption of sensitive data is definitely an issue.
Cheers!
-
I haven't finished SCP which was originally contributed by David Harrigan. I cannot commit a lot of time at the moment for SCP so it may or may not land in 19.1. All hands welcome.
Cheers,
Franco
-
I'm interested in looking into this - where (what branch?) can I have a look in?
-
First, go to plugins and in issues and PR's search for scp. You'll find the correct ones with all the discussions around. I think this would be the best start :)
-
The unfinished feature is in the development version under System: Configuration: Backups: Secure Copy.
The development version is the master branch of core.git.
Cheers,
Franco
-
On that same note of backup/restore, I just had to do one b/c I swapped out the SSD that I had originally deployed on the firewall for a "real" (higher quality) one.
I noticed that the configuration file doesn't appear to contain information about the plugins installed, or the version. I realize why this is - so the configuration can be restored to an old(er) installation and still work, or be imported vertically to a newer one.
However, what I don't see anywhere is the means to actually run a "reproducible backup" where one can quickly, in "just a few clicks", restore the firewall to its prior state including all plugins, firewall version, packages, etc.
With this type of backup, the user would be able to restore a firewall to exactly the same state as it was before. For example, at the start of the restore process the UI can ask if the user also wants to perform a roll-forward (or back, however the case may be) to the "installation state" (OPNSense version, packages, plugins) described by the configuration file (and, of course, show that state).
Does that make sense? Does that feature exist, and is it something that might be desirable?
-
https://github.com/opnsense/core/issues/1663
It's half-working. But we don't want to auto-install plugins, just provide a hint and maybe a "reinstall all missing plugins" button.
With any backup, update to the latest version prior to restore or restore first and swiftly upgrade to the latest version before use.
Cheers,
Franco
-
Right. I understand why you wouldn't auto-install stuff, and that's definitely NOT what I'm suggesting.
However, you could give the user the option to accelerate restoring that bit, i.e. "Do you also want to update the installation as described in this configuration?" (and describe the updates), as well as "Please select which plugins you wish to restore" (and present a list of the plugins selected from which the user might untick the ones he doesn't want to restore)...
Optionally, once restored and on reboot (with all the above done), also compare the list of packages installed to the list of packages included in the restored configuration and tell the user "The following packages which you installed manually are still missing, do you wish for them to be installed now?"
Does that sound more sensible? I guess what I'm looking for is a means to restore the entire firewall to its backed-up state, to make it as painless as possible if I'm starting from scratch with fresh hardware (for example). I realize there are modifications users can do which would make this more difficult, but we shouldn't anticipate/account for those just yet. For now, something like the above would do very nicely.
Later on we can decide if we add a mechanism that actually tracks filesystem-level deviations from the base install and backs those up (or at least documents them for the user).