On upgrade duckdb unsupported version.

[Com_DAC]

The upgrade went fine on my test environment instance so I figured I'd try it on my main instance and got the following when upgrading:

Code: [Select]

***GOT REQUEST TO UPGRADE***
Currently running OPNsense 23.7.12_5 at Tue Jan 30 09:41:44 EST 2024
Fetching packages-24.1-amd64.tar: ......................................... done
Fetching base-24.1-amd64.txz: .......... done
Fetching kernel-24.1-amd64.txz: ..... done
Extracting packages-24.1-amd64.tar... done
Extracting base-24.1-amd64.txz... done
Extracting kernel-24.1-amd64.txz... done
Please reboot.
>>> Invoking upgrade script 'squid-plugin.php'
Squid web proxy is not active. Not installing replacement plugin.
>>> Invoking upgrade script 'unbound-duckdb.py'
Traceback (most recent call last):
  File "/usr/local/opnsense/site-python/duckdb_helper.py", line 65, in __enter__
    self.connection = duckdb.connect(database=self._path, read_only=self._read_only)
duckdb.IOException: IO Error: Trying to read a database file with version number 39, but we can only read version 51.
The database file was created with DuckDB version v0.6.0 or v0.6.1.

The storage of DuckDB is not yet stable; newer versions of DuckDB cannot read old database files and vice versa.
The storage will be stabilized when version 1.0 releases.

For now, we recommend that you load the database file in a supported version of DuckDB, and use the EXPORT DATABASE command followed by IMPORT DATABASE on the current version of DuckDB.

See the storage page for more information: https://duckdb.org/internals/storage

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/etc/rc.syshook.d/upgrade/20-unbound-duckdb.py", line 41, in <module>
    if export_database('/var/unbound/data/unbound.duckdb', '/var/cache/unbound.duckdb', 'unbound', 'unbound'):
  File "/usr/local/opnsense/site-python/duckdb_helper.py", line 147, in export_database
    with DbConnection(source, read_only=True) as db:
  File "/usr/local/opnsense/site-python/duckdb_helper.py", line 75, in __enter__
    raise StorageVersionException(str(e))
duckdb_helper.StorageVersionException: IO Error: Trying to read a database file with version number 39, but we can only read version 51.
The database file was created with DuckDB version v0.6.0 or v0.6.1.

The storage of DuckDB is not yet stable; newer versions of DuckDB cannot read old database files and vice versa.
The storage will be stabilized when version 1.0 releases.

For now, we recommend that you load the database file in a supported version of DuckDB, and use the EXPORT DATABASE command followed by IMPORT DATABASE on the current version of DuckDB.

See the storage page for more information: https://duckdb.org/internals/storage
>>> Error in upgrade script '20-unbound-duckdb.py'
***DONE***
Is there a manual way to force the db to upgrade to a supported version?

Thank you

[MCMLIX]

I'm also getting the same error.
I have never had any errors in a upgrade before.

[franco]

Kinda odd, because 23.7 introduced duckdb 0.8.x. Stray database must have been broken for a while now.

https://github.com/opnsense/changelog/blob/930133dafada3dbbe5bb63ffd5e5f4d9ecd61437/community/23.7/23.7#L112


Cheers
Franco

[Com_DAC]

I did a clean install importing the existing config and it all went well. The only downside is this will have me visit all the other sites that I maintain so I'm on site when trying the upgrade.

Thanks.

[amichel]

Same here, I hope this is getting fixed, workarounded so the upgrade is doable without getting onsite.

[franco]

You can just drop the database under Reporting: Settings: Reset DNS data.

Mind you the choice that the upgrade exits here is deliberate, because it runs into an error condition and requires manual intervention by the user.


Cheers,
Franco

[amichel]

Thank you Franco, this might be a smarter approach than my solution.
I simply renamed the database though the shell and then upgrade worked like a charm. Will delete it now.
 

[Com_DAC]

Thank you Franco. I did the Reset DNS data option and was able to upgrade two remote sites without issue.

[franco]

Yay

We will make this a bit simpler in the future in cases where the reporting is turned off and left a bad database behind. Then we can auto-clean this.


Cheers,
Franco

[pmcgleenon]

I had a similar issue with unbound DNS + reporting enabled
It was fixed by Reporting: Settings: Reset DNS data.

Just reporting it here since it's different from the unsupported DB error message

***GOT REQUEST TO UPGRADE***
Currently running OPNsense 23.7.12_5 at Thu Feb  1 07:31:55 GMT 2024
Fetching packages-24.1-amd64.tar: ................ done
Fetching base-24.1-amd64.txz: .... done
Fetching kernel-24.1-amd64.txz: ... done
Extracting packages-24.1-amd64.tar... done
Extracting base-24.1-amd64.txz... done
Extracting kernel-24.1-amd64.txz... done
Please reboot.
>>> Invoking upgrade script 'squid-plugin.php'
Squid web proxy is not active. Not installing replacement plugin.
>>> Invoking upgrade script 'unbound-duckdb.py'
Abort trap (core dumped)
>>> Error in upgrade script '20-unbound-duckdb.py'
***DONE***

[franco]

> Abort trap (core dumped)

Pretty hard error and not traceable, but indeed the right solution to drop the database.

Thanks for the report. These pre-upgrade hooks are pretty neat and appear to protect users from harm going forward (blocking updates until issues have been resolved if they can't be resolved by the scripts).

One of these days I want to add one that checks for available disk space beforehand.


Cheers,
Franco

[pmcgleenon]

Yes it was pretty cool that the upgrade didn't proceed!

I'm guessing the abort came from export_database in duck_db_helper

https://github.com/opnsense/core/blob/d8ba131aadcceb2dd9719627a1363b34aad41e70/src/etc/rc.syshook.d/upgrade/20-unbound-duckdb.py#L45

Maybe the upgrade script should just drop the db instead of trying to export it

[opn_minded]

I had a similar issue with unbound DNS + reporting enabled
It was fixed by Reporting: Settings: Reset DNS data.

Just reporting it here since it's different from the unsupported DB error message

***GOT REQUEST TO UPGRADE***
Currently running OPNsense 23.7.12_5 at Thu Feb  1 07:31:55 GMT 2024
Fetching packages-24.1-amd64.tar: ................ done
Fetching base-24.1-amd64.txz: .... done
Fetching kernel-24.1-amd64.txz: ... done
Extracting packages-24.1-amd64.tar... done
Extracting base-24.1-amd64.txz... done
Extracting kernel-24.1-amd64.txz... done
Please reboot.
>>> Invoking upgrade script 'squid-plugin.php'
Squid web proxy is not active. Not installing replacement plugin.
>>> Invoking upgrade script 'unbound-duckdb.py'
Abort trap (core dumped)
>>> Error in upgrade script '20-unbound-duckdb.py'
***DONE***

same here, confirmation: resetting dns data worked.

[tokar86a]

You can just drop the database under Reporting: Settings: Reset DNS data.

Mind you the choice that the upgrade exits here is deliberate, because it runs into an error condition and requires manual intervention by the user.


Cheers,
Franco

I ended up whit the same problem and this fix it. thanks.

[nzkiwi68]

I had the same error - fixed by resetting DNS data

• Reporting > Settings > Unbound DNS reporting "Reset DNS data"

Then do the upgrade again - worked flawlessly.