Automatic, periodic configuration backup via SCP

[drivera]

Hi!

I found a GitHub issue report (https://github.com/opnsense/plugins/pull/458) which suggests that automatic, periodic configuration backup via SCP has already been implemented and merged into master since January, but I've found nothing relating to it within the current release's UI (18.7.6).

Also, the plugin itself (scp-backup) seems to have disappeared, further suggesting that this got added to core as the above link suggests.

However, I can't find the configuration page anywhere.  Did this get scrapped? Did it get pushed out to the 19.X release tree?

This is important functionality for those of us who need to keep backups locally (due to company policies, for instance). I'd settle for a backup via a network mount (NFS? Samba isn't ideal...but better than nothing)...

I can code this manually, but I'd just as soon have a UI to administer it, for consistency's sake (also, it's easier to backup a single config XML that has everything, including its "self-backup" configuration).

Thanks!

[mimugmail]

When you have master installed you can see this section on the backup page where also gmail and nextcloud sits.

But I'm quite sure it's not working .. just test it

[drivera]

I'm pretty sure I have master installed - rather: I installed from the USB drive image (vga) - is this not based on master? Further, I ran updates to 18.7.6 ... would that not include that functionality?

Am I misunderstanding how OPNSense releases are built/structured? (if so, can you point me to where I might get up to speed? :D)

Thanks for the quick replies!

[mimugmail]

No, you dont. "master" is the current development branch. There are also branches for 18.7 and so on .. when someone sends a fix it lands in master. When it's verified to work it'll be merged to 18.7 branch and get in the latest release.

On a test machine (and only on a test machine) go to CLI:

opnsense-update -t opnsense-devel
opnsense-code core
cd /usr/core && make upgrade


Then you are on master :)

[drivera]

Ok... I guess what I meant to ask was "this was merged in January - shouldn't it have made the stable/prod branch by now?" :D

But you're 100% correct.

So I guess this feature never made it to the 18.7 branch (since 18.7.6 only came a few days ago)?

Thanks!

[mimugmail]

Yep, I'm quite sure there's something missing which prevents adding it to stable.

[fabian]

FYI. It is there but it currently has no implementation in the class is empty, so it is not operational:
https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/library/OPNsense/Backup/Scp.php#L115

The core class is what Franco wanted to move over from the plugins repository but did never finish it so it is still a stub and also won't run on alpha / beta even if visible there.

The plugin should be functional but it is not intended for release (should be moved to core).

[drivera]

I can probably take a look at that as well.

[drivera]

Another interesting thought: backup to remote GIT (SVN?)

Still has security implications with all the unencrypted stuff that goes in that XML, which kinda begs the question: doesn't it work better to just encrypt the "sensitive" stuff vs. (optionally?) the whole thing? It would make life simpler with regards to change tracking with something like GIT/SVN as a backup mechanism.

Thoughts?

[fabian]

GIT may produce a disk space issue and encrypted is against the principle of git (maintain the diffs).
Also I would need something like https://github.com/libgit2/rugged to write such an adapter because I would have to access the repository programmatically.

[drivera]

Well, since the intent would be to back up to an external GIT repository, and it'd only be config.xml, and I'm fairly sure GIT has the means to reduce local space consumption by trimming unnecessary history, this avenue might actually turn out to not be a bad idea.

I'll try to think up an approach and see if I can come up with a coherent proposal for it.  I see value there, but you're right: the encryption of sensitive data is definitely an issue.

Cheers!

[franco]

I haven't finished SCP which was originally contributed by David Harrigan. I cannot commit a lot of time at the moment for SCP so it may or may not land in 19.1. All hands welcome.


Cheers,
Franco

[drivera]

I'm interested in looking into this - where (what branch?) can I have a look in?

[mimugmail]

First, go to plugins and in issues and PR's search for scp. You'll find the correct ones with all the discussions around. I think this would be the best start :)

[franco]

The unfinished feature is in the development version under System: Configuration: Backups: Secure Copy.

The development version is the master branch of core.git.


Cheers,
Franco