[SOLVED] Upgrade from 19.1 fails

[gunnarf]

I've tried several times now, to upgrade my 19.1 to 19.7 I unlock the upgrade and push the Upgrade now button. It then starts to download, reboot, and when I log in again I'm still on 19.1

I've also tried upgrading from the console and choose 19.7 when appropriate. Same result, after reboot still running 19.1

When  looking in updates it looks like this:

Package Name   Current Version   New Version   Required Action
base                   19.7.3          19.1.8          upgrade

Which is plain nonsense.

In the Dashboard it says:

Versions   OPNsense 19.1.10_1-amd64
FreeBSD 11.2-RELEASE-p14-HBSD
OpenSSL 1.0.2s 28 May 2019

[rainerle]

Can you try that from the console menu? Item 12.

And then post the console output here. Using SSH should make getting the output easier...

[gunnarf]

This is what I get when updating from the console:

Enter an option: 12

Fetching change log information, please wait... done

This will automatically fetch all available updates, apply them,
and reboot if necessary.

A major firmware upgrade is available for this installation: 19.7

Make sure you have read the release notes and migration guide before
attempting this upgrade.  Around 400MB will need to be downloaded and
require 800MB of free space.  Continue with this major upgrade by
typing the major upgrade version number displayed above.

Minor updates may be available, answer 'y' to run them instead.

Proceed with this action? [19.7/y/N]: 19.7

Fetching packages-19.7-OpenSSL-amd64.tar: ................................................... done
Fetching base-19.7-amd64.txz: .......... done
Fetching kernel-19.7-amd64.txz: ..... done
!!!!!!!!!!!! ATTENTION !!!!!!!!!!!!!!!
! A critical upgrade is in progress. !
! Please do not turn off the system. !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Extracting packages-19.7-OpenSSL-amd64.tar... done
Extracting base-19.7-amd64.txz... done
Installing kernel-19.7-amd64.txz... done
Please reboot.
>>> Invoking stop script 'beep'
>>> Invoking stop script 'freebsd'
>>> Invoking stop script 'backup'
>>> Invoking backup script 'captiveportal'
>>> Invoking backup script 'dhcpleases'
>>> Invoking backup script 'duid'
>>> Invoking backup script 'netflow'
Cannot 'stop' flowd_aggregate. Set flowd_aggregate_enable to YES in /etc/rc.conf or use 'onestop' instead of 'stop'.
Cannot 'start' flowd_aggregate. Set flowd_aggregate_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.
>>> Invoking backup script 'rrd'
>>> Invoking stop script 'config'
Shutdown NOW!
shutdown: [pid 78283]

*** FINAL System shutdown message from root@OPNsense.gflygt.se ***

System going down IMMEDIATELY



System shutdown time has arrived

And after update I get this on the console:

root@OPNsense:~ # uname -a
FreeBSD OPNsense.gflygt.se 11.2-RELEASE-p14-HBSD FreeBSD 11.2-RELEASE-p14-HBSD  07680caafe9(stable/19.7)  amd64

Witch seems ok.

BUT look at what the login says after the update even though uname -a says 19.7

eldorado:~ gunnar$ ssh root@192.168.1.1
Password for root@OPNsense.gflygt.se:
Last login: Fri Nov  1 15:56:24 2019 from 192.168.1.32
----------------------------------------------
|      Hello, this is OPNsense 19.1          |         @@@@@@@@@@@@@@@
|                                            |        @@@@         @@@@
| Website:   https://opnsense.org/        |         @@@\\\   ///@@@
| Handbook:   https://docs.opnsense.org/   |       ))))))))   ((((((((
| Forums:   https://forum.opnsense.org/  |         @@@///   \\\@@@
| Lists:   https://lists.opnsense.org/  |        @@@@         @@@@
| Code:      https://github.com/opnsense  |         @@@@@@@@@@@@@@@
----------------------------------------------

*** OPNsense.gflygt.se: OPNsense 19.1.10_1 (amd64/OpenSSL) ***        <<<<<<<THIS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 LAN (igb1)      -> v4: 192.168.1.1/24
                    v6: 2001:470:28:277::1/64
 OPT1 (igb2)     -> v4: 192.168.53.1/24
 TunnelBroker (gif0) -> v6: 2001:470:27:277::2/128
 WAN (igb0)      -> v4/DHCP4: 85.225.24.248/24

 HTTPS: SHA256 E2 36 FD FB 7B 4A 90 14 14 81 9B 8B 6C D1 7B 2A
               DC 5B 45 81 97 7A E0 84 4A 62 F1 B5 42 2C 32 4A
 SSH:   SHA256 2Cbsm/aYab4229xIIiwDfKpSMGkkamdzqn4HtRJdPVo (ECDSA)
 SSH:   SHA256 uXDwNYMkA5/dfBIOt2YMRs0n9tQFmX2cY7DJs948i/w (ED25519)
 SSH:   SHA256 oZbioXhle23p0HOGpbaLg81JYaEhLYRZZwyeSlf+ilQ (RSA)

  0) Logout                              7) Ping host
  1) Assign interfaces                    Shell
  2) Set interface IP address            9) pfTop
  3) Reset the root password            10) Firewall log
  4) Reset to factory defaults          11) Reload all services
  5) Power off system                   12) Update from console
  6) Reboot system                      13) Restore a backup

Enter an option:


So still on 19.1 and when logging in to the web interface it says the same.

If I go to update firmware in the web interface and presses update it starts to download 19.1 !!!

[franco]

Hello,

Your packages are wedged for one reason or another: base and kernel upgrade fine but if the upgrade of the packages stalls (manual install, file system issue, etc.) you're not going to get further.

Please provide the contents of /var/cache/opnsense-update/.upgrade.log which will reveal why your packages won't upgrade.


Cheers,
Franco

[gunnarf]

I attach the log file

[franco]

Your cached updates are gone after reboot:

Updating OPNsense repository catalogue...
repository OPNsense has no meta file, using default settings
Unable to update repository OPNsense
Error updating repositories!

I have no idea what that means. They should be persistently stored. Maybe you don't have enough space on the device left?


Cheers,
Franco

[gunnarf]

I'll try to send you another update.log now. I just ran a update from the console and uname -a looks like:

root@OPNsense:~ # uname -a                                                      FreeBSD OPNsense.gflygt.se 11.2-RELEASE-p14-HBSD FreeBSD 11.2-RELEASE-p14-HBSD  07680caafe9(stable/19.7)  amd64

[franco]

Same issue... can you just run this package update command from the shell and see what error happens *before* reboot?

# opnsense-update -pur 19.7


Cheers,
Franco

[gunnarf]

If I run pkg update I get the following:

root@OPNsense:~ # uname -a
FreeBSD OPNsense.gflygt.se 11.2-RELEASE-p14-HBSD FreeBSD 11.2-RELEASE-p14-HBSD  07680caafe9(stable/19.7)  amd64
root@OPNsense:~ # pkg update
Updating OPNsense repository catalogue...
pkg: Repository OPNsense load error: access repo file(/var/db/pkg/repo-OPNsense.sqlite) failed: No such file or directory
Fetching meta.txz: 100%    1 KiB   1.5kB/s    00:01
Fetching packagesite.txz: 100%  175 KiB 178.9kB/s    00:01
Processing entries: 100%
OPNsense repository update completed. 689 packages processed.
All repositories are up to date.

So there is an issue with sqlite obviously.

root@OPNsense:~ # ls -l /var/db/pkg/repo-OPNsense.sqlite
-rw-r--r--  1 root  wheel  1384448 Jul 13 11:04 /var/db/pkg/repo-OPNsense.sqlite

and for showing how much space I've got:

root@OPNsense:~ # df -h
Filesystem         Size    Used   Avail Capacity  Mounted on
/dev/gpt/rootfs     13G    1.4G     11G    11%    /
devfs              1.0K    1.0K      0B   100%    /dev
devfs              1.0K    1.0K      0B   100%    /var/unbound/dev
devfs              1.0K    1.0K      0B   100%    /var/dhcpd/dev

[franco]

Let's try the hard way...

# opnsense-bootstrap -b -r 19.7
# pkg upgrade -f


Cheers,
Franco

[gunnarf]

So far ran the first command, and no reboot occurred

root@OPNsense:~ # opnsense-bootstrap -b -r 19.7
This utility will attempt to turn this installation into the latest
OPNsense 19.7 release.  All packages will be deleted, the base
system and kernel will be replaced, and if all went well the system
will automatically reboot.

Proceed with this action? [y/N]: y

Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
Installing pkg-1.12.0...
package pkg is already installed, forced install
Extracting pkg-1.12.0: 100%
Updating FreeBSD repository catalogue...
Fetching meta.txz: 100%    944 B   0.9kB/s    00:01
Fetching packagesite.txz: 100%    6 MiB 951.2kB/s    00:07
Processing entries: 100%
FreeBSD repository update completed. 32783 packages processed.
All repositories are up to date.
Updating database digests format: 100%
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
   ca_root_nss: 3.44.1 -> 3.47

Number of packages to be upgraded: 1

287 KiB to be downloaded.
[1/1] Fetching ca_root_nss-3.47.txz: 100%  287 KiB 293.5kB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Upgrading ca_root_nss from 3.44.1 to 3.47...
[1/1] Extracting ca_root_nss-3.47: 100%
fetch: https://github.com/opnsense/core/archive/stable/19.7.tar.gz: size of remote file is not known
/tmp/opnsense-bootstrap/core.tar.gz                   7643 kB 2155 kBps 00m03s
/tmp/opnsense-bootstrap/core-stable-19.7/Scripts/version.sh: git: not found
make: "/tmp/opnsense-bootstrap/core-stable-19.7/Makefile" line 31: warning: "/tmp/opnsense-bootstrap/core-stable-19.7/Scripts/version.sh" returned non-zero status

Now for the pkg upgrade -f

added as a file

[gunnarf]

Yes!

Versions   OPNsense 19.7.6-amd64
FreeBSD 11.2-RELEASE-p14-HBSD
OpenSSL 1.0.2t 10 Sep 2019

from the dashboard, and from the console:

*** OPNsense.gflygt.se: OPNsense 19.7.6 (amd64/OpenSSL) ***

 LAN (igb1)      -> v4: 192.168.1.1/24
                    v6: 2001:470:...
 OPT1 (igb2)     -> v4: 192.168.53.1/24
 OPT3 (ovpns1)   -> v4: 10.10.0.1/32
 TunnelBroker (gif0) -> v6: 2001:470:...
 WAN (igb0)      -> v4/DHCP4: 85.225...

 
  0) Logout                              7) Ping host
  1) Assign interfaces                    Shell
  2) Set interface IP address            9) pfTop
  3) Reset the root password            10) Firewall log
  4) Reset to factory defaults          11) Reload all services
  5) Power off system                   12) Update from console
  6) Reboot system                      13) Restore a backup

Enter an option:

The hard way, was the right way it seems

And all of a sudden my default gateway for ipv6 was set by the system itself. I had to set it manually after reboot before.

[franco]

Nice, looking good now.


Cheers,
Franco