Backup API

[edz]

The below script was working on 24.1.x to retrieve opnsense backup via the api but now fails with HTTP/2 500 error. The user has GUI Diagnostics: Configuration History and GUI Firewall: Aliases privileges.

Code: [Select]

#!/usr/bin/bash

# Change API key and secret, number of days to keep backups, the path to your backups and the hostname for your firewall

key="redacted"
secret="redacted"
daystokeep=90
destination="backup/path"
fwhost="opnsense.hostnameredacted.com"

date=$(date +%Y-%m-%d)

result=$(/usr/bin/curl -I -s -k -u "$key":"$secret" https://$fwhost/api/core/backup/download/this | head -1)

if [[ $result != *"200"* ]]; then
   echo "Result of the HTTP request is $result"
      exit 1
      fi

      /usr/bin/curl -s -k -u "$key":"$secret" https://$fwhost/api/core/backup/download/this > $fwhost-config-$date.xml

      error=$?

Code: [Select]

./opnsense_backup_v2API.sh
Result of the HTTP request is HTTP/2 500

[sined]

have you found a solution?
I have the problem, no matter what I do I always get "No input file specified." back

[edz]

Hi team,

The Backup API is still returning an error in the final 24.7 release.  The script posted above returns "Result of the HTTP request is HTTP/2 500"

and the script below returns "{"errorMessage":"Unexpected error, check log for details"}%" but I do not see any logs in the UI when viewing System>Log Files. Happy to keep on digging around if someone can assist in how to trouble shoot.

Code: [Select]

#!/bin/bash

KEY="partiallyredacted6Raif/mEENSbm8"
SECRET="partiallyredacted6RaifsInpBlickOeUCZcf"
HOST="opnsense.redacted.com"
PATHCONFIG="/Downloads/opnsense"
DATE=$(date +%Y%m%d)

curl -u "${KEY}":"${SECRET}" https://"${HOST}"/api/core/backup/download/this --create-dirs -o "$PATHCONFIG"/opnsense-config-"${DATE}".xml


Code: [Select]

❯ cat opnsense-config-20240726.xml
{"errorMessage":"Unexpected error, check log for details"}%
and here the is the verbose output

Code: [Select]

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*
Host opnsense.redacted.com:443 was resolved.
* IPv6: (none)
* IPv4: [myIP]
*   Trying [myIP]:443...
* Connected to opnsense.redacted.com (myIP) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [327 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [19 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [2829 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [520 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.redacted.com
*  start date: Jul 10 13:02:21 2024 GMT
*  expire date: Oct  8 13:02:20 2024 GMT
*  subjectAltName: host "opnsense.redacted.com" matched cert's "*.redacted.com"
*  issuer: C=US; O=Let's Encrypt; CN=R10
*  SSL certificate verify ok.
* using HTTP/2
* Server auth using Basic with user 'partiallyredacted6Raif/mEENSbm8'
* [HTTP/2] [1] OPENED stream for https://opnsense.redacted.com/api/core/backup/download/this
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: opnsense.redacted.com]
* [HTTP/2] [1] [:path: /api/core/backup/download/this]
* [HTTP/2] [1] [authorization: Basic K3FpcmNSOUZ2YXlKOWUrOTJTNVJaZUZucW1ETY2Y=]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET /api/core/backup/download/this HTTP/2
> Host: opnsense.redacted.com
> Authorization: Basic K3FpcmNSOUZ2YXlKOWUrOTJTNVJaZUZucW1ET0UNaY2Y=
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/2 500
< set-cookie: PHPSESSID=3b69be9b49af90e42a7a16b3713f9546; path=/; secure; HttpOnly
< expires: Thu, 19 Nov 1981 08:52:00 GMT
< cache-control: no-store, no-cache, must-revalidate
< pragma: no-cache
< content-type: application/json;charset=utf-8
< strict-transport-security: max-age=31536000
< date: Fri, 26 Jul 2024 10:49:06 GMT
< server: OPNsense
<
{ [58 bytes data]
100    58    0    58    0     0    865      0 --:--:-- --:--:-- --:--:--   878
* Connection #0 to host opnsense.redacted.com left intact

[franco]

https://github.com/opnsense/core/commit/897d3cce3

# opnsense-patch 897d3cce3


Cheers,
Franco

[sined]

In the meantime I had managed it with the script. My problem was that I did not access the web interface via 80 or 443, but via a special port.

Today I updated to 24.7_5 and ran directly into the backup problem again. But with the adjustment of the file as in the post above me, the backup worked again immediately.

Presumably the file will be adapted for everyone in one of the next updates. Will my file then also be overwritten again or does it check during the update whether it needs to change the file?

Translated with DeepL.com (free version)

[edz]

Thanks, Franco!

It is now working with this patch applied

https://github.com/opnsense/core/commit/897d3cce3

# opnsense-patch 897d3cce3


Cheers,
Franco

[franco]

Thanks, will be hotfixed on Monday.


Cheers,
Franco

[pyrodex]

Thanks, will be hotfixed on Monday.


Cheers,
Franco

I have this same issue and using the same patch but still get the errors in the UI:

Code: [Select]

[26-Jul-2024 14:27:41 America/New_York] ArgumentCountError: Too few arguments to function OPNsense\Mvc\Response::setContentType(), 1 passed in /usr/local/opnsense/mvc/app/controllers/OPNsense/Core/Api/BackupController.php on line 200 and exactly 2 expected in /usr/local/opnsense/mvc/app/library/OPNsense/Mvc/Response.php:64
Stack trace:
#0 /usr/local/opnsense/mvc/app/controllers/OPNsense/Core/Api/BackupController.php(200): OPNsense\Mvc\Response->setContentType('application/oct...')
#1 /usr/local/opnsense/mvc/app/library/OPNsense/Mvc/Dispatcher.php(165): OPNsense\Core\Api\BackupController->downloadAction('this')
#2 /usr/local/opnsense/mvc/app/library/OPNsense/Mvc/Router.php(156): OPNsense\Mvc\Dispatcher->dispatch(Object(OPNsense\Mvc\Request), Object(OPNsense\Mvc\Response), Object(OPNsense\Mvc\Session))
#3 /usr/local/opnsense/mvc/app/library/OPNsense/Mvc/Router.php(139): OPNsense\Mvc\Router->performRequest(Object(OPNsense\Mvc\Dispatcher))
#4 /usr/local/opnsense/www/api.php(36): OPNsense\Mvc\Router->routeRequest('/api/core/backu...', Array)
#5 {main}

Do I need to restart anything?

My script:

Code: [Select]

#!/bin/bash

# Since we need curl let's see if it is installed
if cmd=$(command -v curl); then
        CURL=$cmd
else
        echo "$(date "+[%Y-%m-%d - %H:%M:%S%p]"): ERROR: Required binary curl is missing, please install it" > $LOG 2>&1
        exit 1
fi

# Slack notification function
notify () {
        message=$1
        curl -s https://api.pushover.net/1/messages.json -d "token=${apitoken}&user=${userkey}&title=OPNSense%20Backup&message=${message}&timestamp=${ts}&device=${device}"
        echo "$message" | mailx -r opnsense-backup@domain.com -s "OPNsense Backup" user@domain.com
}

# Variables
PINGHOST=192.168.xx.1
HOST=$PINGHOST
PFPATH=/mnt/scripts/opnsense
ts=$(date +%s)

# api access
# Backup user
key="redacted"
secret="redacted"

# pushover settings
userkey="redacted"
apitoken="redacted"
device="redacted"

# Check to make sure firewall is up
ping -c 1 -W 5 $PINGHOST &>/dev/null
if [ $? -ne 0 ] ; then
   echo "$PINGHOST is down! bailing"
   notify "$PINGHOST is down! Check to make sure firewall is online"
   exit
fi

# Clean up old files
find $PFPATH -name "*.xml" -mtime +7 -exec rm -rf {} \;

# Download the config
# Pre 23.7.8, this is now native in the API
#/usr/bin/curl -k -u "$key":"$secret" https://$HOST/api/backup/backup/download -o $PFPATH/$(/bin/date +%Y%m%d).xml
/usr/bin/curl -k -u "$key":"$secret" https://$HOST/api/core/backup/download/this -o $PFPATH/$(/bin/date +%Y%m%d).xml

# Check if the configuration is the right size...
XML_CHECK=$(grep -c "?xml" $PFPATH/`date +%Y%m%d`.xml)
SIZE=$(stat -c %s $PFPATH/`date +%Y%m%d`.xml)


# File must be zero size...
if [ $SIZE -eq 0 ]; then
        notify "Configuration download was ZERO bytes, check the backup process!"
        exit
fi
if [ $XML_CHECK -eq 0 ]; then
        notify "Configuration download did not meet expected XML format, check the backup process!"
        exit
fi
notify "Backup of Firewall completed on $(date)"

I run my script from a Debian 12 host and it worked just fine in 24.1 for every release.

[pyrodex]

So the instructions need to include restarting the webGUI AFTER applying the patch. After doing that I am able to backup now.

[franco]

> So the instructions need to include restarting the webGUI AFTER applying the patch. After doing that I am able to backup now.

I can't say why it would. And if you want fool-proof instructions just reboot after opnsense-patch


Cheers,
Franco