[SOLVED] upgrade from 20.1.9 to 20.7 failed

[alex_rhys-hurn]

Hi,

I have a freshly installed hardware firewall (Dell Server). I installed and configured 20.1.9.

A couple of days later the new 20.7 was released. I checked for updates, and unlocked the 20.7 upgrade, and proceeded to upgrade from GUI.

Something was downloaded and the firewall rebooted. "EDIT: Really a very poor report of the problem by me. Sorry. I didnt pay proper attention, and left the thing to run while I went for "coffee", well it was the weekend."

Now, its stuck on 20.1.9_1 with following errors:

Code: [Select]

# opnsense-update
Updating OPNsense repository catalogue...
pkg-static: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/latest/meta.txz: Not Found
repository OPNsense has no meta file, using default settings
pkg-static: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/latest/packagesite.txz: Not Found
Unable to update repository OPNsense
Error updating repositories!
root@gw:/home/alex_rhys-hurn #

Why is it still looking for 20.1 files and not 20.7?

When I try that URL on my laptop web browser, I get File Not Found.

I notice that the URL says FreeBSD:12 and then Opnsense 20.1 That cant be right.

If I try to download https://pkg.opnsense.org/FreeBSD:12:amd64/20.7/latest/packagesite.txz manually I get a file.

How can I manually update this URL or reset the updates database so it can download afresh?

Help appreciated, thanks,

Alex

[alex_rhys-hurn]

Here is some debug output from pkg:

Code: [Select]

root@gw:~ # pkg -d update -f
DBG(1)[4764]> pkg initialized
Updating OPNsense repository catalogue...
DBG(1)[4764]> PkgRepo: verifying update for OPNsense
DBG(1)[4764]> PkgRepo: need forced update of OPNsense
DBG(1)[4764]> Pkgrepo, begin update of '/var/db/pkg/repo-OPNsense.sqlite'
DBG(1)[4764]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/latest/meta.txz with opts "i"
pkg: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/latest/meta.txz: Not Found
repository OPNsense has no meta file, using default settings
DBG(1)[4764]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/latest/packagesite.txz with opts "i"
pkg: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/latest/packagesite.txz: Not Found
Unable to update repository OPNsense
Error updating repositories!
root@gw:~ #


[alex_rhys-hurn]

Hmmm....

Very odd. The contents of the repo file were:

Code: [Select]

# cat OPNsense.conf
OPNsense: {
  fingerprints: "/usr/local/etc/pkg/fingerprints/OPNsense",
  url: "pkg+https://pkg.opnsense.org/${ABI}/20.1/latest",
  signature_type: "fingerprints",
  mirror_type: "srv",
  priority: 11,
  enabled: yes
}

So I edited the line to   url: "pkg+https://pkg.opnsense.org/${ABI}/20.7/latest",

ran pkg update which produced (which is good):

Code: [Select]

pkg update -f
Updating OPNsense repository catalogue...
Fetching meta.txz: 100%    1 KiB   1.5kB/s    00:01   
Fetching packagesite.txz: 100%  186 KiB  47.6kB/s    00:04   
Processing entries: 100%
OPNsense repository update completed. 704 packages processed.
All repositories are up to date.

I then triggered option 12 from the root menu to update which produced (which also looks good):

Code: [Select]

Enter an option: 12

Fetching change log information, please wait... fetch: https://pkg.opnsense.org/FreeBSD:12:amd64/20.1/sets/changelog.txz.sig: Not Found

This will automatically fetch all available updates, apply them,
and reboot if necessary.

Proceed with this action? [y/N]: y

Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking for upgrades (1 candidates): . done
Processing candidates (1 candidates): . done
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
opnsense: 20.1.9_1 -> 20.7

Number of packages to be upgraded: 1

4 MiB to be downloaded.
[1/1] Fetching opnsense-20.7.txz: .......... done
Checking integrity... done (0 conflicting)
[1/1] Upgrading opnsense from 20.1.9_1 to 20.7...
[1/1] Extracting opnsense-20.7: .......... done
Stopping configd...done
Resetting root shell
Updating /etc/shells
Unhooking from /etc/rc
Unhooking from /etc/rc.shutdown
opnsense-20.1.9_1: missing file /usr/local/opnsense/firmware-message
opnsense-20.1.9_1: missing file /usr/local/opnsense/firmware-upgrade
Updating /etc/shells
Registering root shell
Hooking into /etc/rc
Hooking into /etc/rc.shutdown
Starting configd.
Keep version OPNsense\Monit\Monit (1.0.8)
Keep version OPNsense\Firewall\Alias (1.0.0)
Keep version OPNsense\OpenVPN\Export (0.0.1)
Keep version OPNsense\CaptivePortal\CaptivePortal (1.0.0)
Keep version OPNsense\Interfaces\Loopback (1.0.0)
Migrated OPNsense\Interfaces\VxLan from 0.0.0 to 1.0.1
Keep version OPNsense\Cron\Cron (1.0.1)
Keep version OPNsense\IPsec\IPsec (0.0.0)
Keep version OPNsense\Backup\NextcloudSettings (1.0.0)
Keep version OPNsense\TrafficShaper\TrafficShaper (1.0.3)
Keep version OPNsense\Syslog\Syslog (1.0.0)
Migrated OPNsense\IDS\IDS from 1.0.3 to 1.0.5
Keep version OPNsense\Proxy\Proxy (1.0.3)
Keep version OPNsense\Diagnostics\Netflow (1.0.1)
Migrated OPNsense\Routes\Route from  <unversioned>  to 1.0.0
Keep version OPNsense\Unboundplus\Miscellaneous (0.0.2)
Keep version OPNsense\Unboundplus\Dnsbl (0.0.1)
Writing firmware setting...done.
Writing trust files...done.
Configuring login behaviour...done.
Configuring system logging...done.
=====
Message from opnsense-20.7:

--
The lion sleeps tonight
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages:

Installed packages to be REMOVED:
sshlockout_pf-0.0.2_2

Number of packages to be removed: 1
[1/1] Deinstalling sshlockout_pf-0.0.2_2...
[1/1] Deleting files for sshlockout_pf-0.0.2_2: .... done
The following package files will be deleted:
/var/cache/pkg/opnsense-20.7-97385e5a92.txz
/var/cache/pkg/opnsense-20.7.txz
The cleanup will free 4 MiB
Deleting files: .. done
All done
Starting web GUI...done.
Generating RRD graphs...done.

All of this was only 4 meg download, and now my dashboard says version 20.7 without any reboot or anything, and took only seconds.

I am rebooting for comfort anyway though.

So, how did I end up with a borked repo file, that tried to merge 20.1 with HBSD 12?

Thoughts appreciated,

Alex

[jkemp]

I have a similar problem. I can't update after failed update. Where did you find the OPNsense.conf file to edit?

[alex_rhys-hurn]

You can find the repo file in:

/usr/local/etc/pkg/repos/OPNsense.conf

Good luck.

[franco]

If packages don't install please post your /var/cache/opnsense-update/.upgrade.log


Thanks,
Franco

[jkemp]

Okay, changing that file fixed it. Great job, that's for responding and posting the remedy.

I'm back up again,

James

[franco]

Nevermind me asking for logs. Maybe in the future I should steer away from asking for debug output or general criticism of "why does this not work and why does nobody fix it".

Anecdote: over a year ago someone complained that we should have logs for major upgrades. Complaints are nice,  but it seems like irony to me now.


Cheers,
Franco

[alex_rhys-hurn]

Dear Franco,

Please don't take any of my "why does it" comments negatively. For me thats always a genuine technical curiosity as to what could have happened and not a frustration that things arent working as expected. This is actually the main reason why I run FOSS software, and its a fun process.

You have helped me directly before (was a newbie and didnt understand the FRR integration), for which I am grateful.

I do know that moderating fora can be a thankless task, so I thank you for doing it.

I didn't manage to upload the file due to time constraints, but here it is now. I haven't looked at it and dont know how useful it is now that the system upgraded ok, I did include some debug output in the original posts, but maybe not enough, so apologies for that.

Reading the other thread about long running "python37-3.7.8_1 and ruby can take ages" I guess I may be a victim of that (prematurely restarting the host) but, as I am running on an old ESXI host of mine with a quad core  Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (4 cores) on a hardware RAID controller with a RAID10 SAS mirror I figured it had had enough time... Lesson learned.

Anyway, thanks again for all the support.

Alex

[franco]

Hi Alex,

No worries. I just sometimes feel help fast can be quite impossible under certain conditions

Looking at the log:

[133/133] Upgrading opnsense from 20.1.9_1 to 20.7...
[133/133] Extracting opnsense-20.7: ........

It looks like it almost finished the final (and most important package in that mirror URL debacle) package but got interrupted. I'm guessing just a few seconds missing.


Cheers,
Franco

[chris42]

Just as an info if someone else looks at this, I got the same error when trying to update to 20.7.
It tried to get packages-20.7-LibreSSL-amd64.tar and then stopped with signature error.

I found that my mirror was set to "(default)". When switching explicitly to Deciso mirror it worked.

[alex_rhys-hurn]

@chris42  I like your solution much better.

I guess when you select or reselct the mirror the repo file gets written again, thereby fixing the issue i noted above. All in all a much safer easier solution than editing the repo file.

Best,

Alex

[ahofmann]

In case someone stumbles across the same problem (can't update because of timeouts) but hasn't found a solution here: for me, it helped to set the MTU of my interfaces to 1472. My setup is OPNSense behind a FritzBox with cable internet.

Edit: it wasn't the mtu, see here: https://forum.opnsense.org/index.php?topic=25725.msg137709#msg137709