OPNsense Forum
English Forums => Tutorials and FAQs => Topic started by: danb35 on July 23, 2020, 12:09:17 pm
-
I just migrated from pfSense to OPNsense. Under pfSense, I had my FreeNAS box running a daily script to download a config file backup from the pfSense box, using the method recommended in their own docs. It was a little messy. OPNsense has the os-api-backup plugin, which makes the process (especially the script) much simpler. However, I didn't see everything pulled together in one place, so here goes:
Assumptions:
- You have a Unix-y machine (Linux, BSD, macOS, maybe even Windows Subsystem for Linux) to run the backup script on
- The WebUI cert on your OPNsense router is trusted on that Unix-y machine
- curl is available on that Unix-y machine
That's really it, so let's get started.
First step, of course, is to install the os-api-backup plugin if it isn't already installed.
Next, you'll want to create a group with limited permissions. In the OPNsense WebUI, go to System -> Access -> Groups and add a new group (I called mine backup). Save the group, then edit it. On the edit screen, under Assigned Privileges, click the edit button, find "Backup API" in the list, and check it. Leave everything else unchecked and click Save. Click Save again to return to the Groups screen.
Now create a user in that group. Go to System -> Access -> Users and add a new user (I called mine, creatively enough, backup_user). I generated a long random password using my password manager, and then discarded it--this user will never log in using that password. Add the user to the backup group and save. Then edit the user, find the API keys heading, and click + to create a new one. This will download a small text file containing an API key and a secret, save it someplace convenient. Click Save to return to the users screen.
That's all you need to do in the OPNsense UI. Now it's time to create the script.
Go to whatever machine you're going to use to run the backup script, fire up your favorite text editor (I like nano--don't judge me), and create the script. Contents are as below:
#!/bin/bash
KEY="api_key"
SECRET="api_secret"
HOST="opnsense_hostname"
PATH="/path/to/backups"
curl -s -k -u $KEY:$SECRET https://$HOST/api/backup/backup/download \
-o $PATH/opnsense-config-$(date +%Y%m%d).xml
find $PATH/ -type f -name '*.xml' -mtime +30 -exec rm {} \;No doubt it's obvious, but edit the variables at the top to match your environment. This will save the backup files with a filename of "opnsense-config-yyyymmdd.xml", and delete everything over 30 days old.
Set up a cron job to run this on your desired schedule, and you're set
-
Figures. I promise, I searched before I posted this, to see if there was already a guide on this, and I didn't find one. But after I post, of course, I see this one:
https://forum.opnsense.org/index.php?topic=15349.0
-
Hi danb35,
I'm just adding this functionality to my infrastructure and I figured out the same solution you posted, it works like a charm.
Do you know if is possible to request an encrypted backup using the APIs? (Passwords are in clear text!)
I see that is possible with the manual download but I don't find anyting regarding the API.
Best
Simone
-
I made some minor tweaks to share. Shouldn't really use PATH as a variable as it can override the system path. escaped some stuff etc.
#!/bin/bash
KEY="api_key"
SECRET="api_secret"
HOST="opnsense_hostname"
PATHCONFIG="/path/to/backups"
DATE=`date +%Y%m%d`
curl -s -u ${KEY}:${SECRET} https://${HOST}/api/backup/backup/download -o $PATHCONFIG/opnsense-config-${DATE}.xml
find ${PATHCONFIG} -type f -name opnsense-config\*.xml -mtime +30 -delete
-
Do you know if is possible to request an encrypted backup using the APIs? (Passwords are in clear text!)
No, it is plaintext only but you can pipe it through openssl to encrypt it.
"-o -" should write it to stdout and "openssl enc <opts>" encrypts it. You can find the options here:
https://www.openssl.org/docs/manmaster/man1/openssl-enc.html
The transfer is secure due to TLS and it makes no sense to encrypt it here since you would transfer the key in the request, so that is not the issue.
-
danb35,
So about duplicating the other instructions, don't sweat it as it is nice to have two sets of instructions. Everyone explains things differently and if I don't understand it in one example I can then look at it in the other. Rather have two working examples and approaches than none.
Oh and yours comes up in DuckDuckGo where the other one does not for some reason.
Thanks
-
Thanks buddy! Works perfect!
-
I took your script as an example and created a version which also encrypts the backups using GPG.
You can find the script here:
https://codeberg.org/SWEETGOOD/andersgood-opnsense-scripts/src/branch/main/backup-opnsense-via-api.sh (https://codeberg.org/SWEETGOOD/andersgood-opnsense-scripts/src/branch/main/backup-opnsense-via-api.sh)
I also wrote a short blogpost in German with some explanations:
https://andersgood.de/kurz-notiert/opnsense-per-api-verschluesseltes-backup-der-konfiguration-anlegen
Thanks to the TS for your work!
-
I took your script as an example and created a version which also encrypts the backups using GPG.
You can find the script here:
https://codeberg.org/SWEETGOOD/andersgood-opnsense-scripts/src/branch/main/backup-opnsense-via-api.sh (https://codeberg.org/SWEETGOOD/andersgood-opnsense-scripts/src/branch/main/backup-opnsense-via-api.sh)
I also wrote a short blogpost in German with some explanations:
https://andersgood.de/kurz-notiert/opnsense-per-api-verschluesseltes-backup-der-konfiguration-anlegen
Thanks to the TS for your work!
Nice script, a minor improvement would be to change the date command output to include hour and minute. This makes it possible to run the backup multiple times a day.
$(date +%Y-%m-%d-%H:%M)
Also, at the end the clean up is a bit dangerous. If you by accident put the backup files in your home or in / the outcome is terrible ;)
-
Thanks for your improvements.
Especially the data rentention command was critical you were absolutely right.
I changed both and commited a new version. Should both be fixed now.
Thanks!
-
I have tried 3 machines now and all of them get this. These are all installed.
./backup-opnsense.sh: line 7: date: command not found
./backup-opnsense.sh: line 7: curl: command not found
./backup-opnsense.sh: line 10: find: command not found
-
Just as a side note: 23.7.6 will offer a base component for config export through the API. I'm not sure where that leaves os-api-backup but it will probably be removed as it's likely redundant and practically unmaintained.
Cheers,
Franco
-
Just as a side note: 23.7.6 will offer a base component for config export through the API. I'm not sure where that leaves os-api-backup but it will probably be removed as it's likely redundant and practically unmaintained.
Cheers,
Franco
@Franco - Is there any documentation or example to guide us through using that new component in 23.7.6? I’d like to start using that method instead.
-
Let me get back on this tomorrow. Need to discuss something first.
Cheers,
Franco
-
I raised a ticket about the (likely missing) ability to pull the latest configuration without listing the name for it first (downloads are timestamped):
https://github.com/opnsense/core/issues/6996
Will report back when this has been implemented.
Cheers,
Franco
-
We will add the following to 23.7.8: https://github.com/opnsense/core/commit/39b531783
The direct download URL is https://xxx.xxx.xxx.xxx/api/core/backup/download/this and I've also changed the EoL message of os-api-backup accordingly.
Cheers,
Franco
-
We will add the following to 23.7.8: https://github.com/opnsense/core/commit/39b531783
The direct download URL is https://xxx.xxx.xxx.xxx/api/core/backup/download/this and I've also changed the EoL message of os-api-backup accordingly.
Cheers,
Franco
Thank you!
-
is there any simple way to package this as a docker container, with a config file to process multiple firewalls (eg a row with parameters related to every remote system)? it will be great :D
-
Mine is slightly different but here's an updated script for those of you who are maybe less technically inclined. Just update the 5 variables at the top to match your configuration and you should be good to go. You'll just need to setup a cron job to run this nightly - the script below will also work run directly from a synology system.
#!/usr/bin/bash
# Change API key and secret, number of days to keep backups, the path to your backups and the hostname for your firewall
key=YOURKEY
secret=YOURSECRET
daystokeep=30
destination="/PATH/TO/SAVE/BACKUPS/TO"
fwhost="IP_ADDRESS_OF_FIREWALL-192.168.1.1"
date=$(date +%Y-%m-%d)
result=$(/usr/bin/curl -I -s -k -u "$key":"$secret" https://$fwhost/api/core/backup/download/this | head -1)
if [[ $result != *"200"* ]]; then
echo "Result of the HTTP request is $result"
exit 1
fi
/usr/bin/curl -s -k -u "$key":"$secret" https://$fwhost/api/core/backup/download/this > $date.xml
error=$?
if [ $error -gt 0 ]; then
echo "Curl returned error number $error"
exit 1
fi
/usr/bin/gzip $date.xml
mv $date.xml.gz $destination
/usr/bin/find $destination/* -mtime +$daystokeep -exec rm {} \;
-
I'd like config plaintext passwords to be encrypted in file. Configs always end up on more locations, backup sites, where permissions vary.