WebGUI and SSH menu lost after upgrade to 24.7

[rackenthogg]

Steps I took:

1. Update to last 24.1 version from WebGUI. Went OK.
2. Upgrade to 24.7 from WebGUI. Seemed to be OK, until reboot. There was only shutdown sound, but then the box hanged, no boot-up sound after half an hour. Also no ping to LAN/WAN interfaces and no WebGUI/SSH access at this moment.
3. I restart the box the hard way (turning the power off//on). There is no boot-up sound. I try to access webGUI - no luck. Luckily there is SSH access. But when I try to run commands from SSH menu, the only response I got is:

Code: [Select]

ld-elf.so.1: /usr/local/bin/php: Undefined symbol "__libc_start1@FBSD_1.7"
ld-elf.so.1: /usr/local/bin/php: Undefined symbol "__libc_start1@FBSD_1.7"

(this line repeated twice).

And this moment there is Internet acess through the box and most of the firewall rules seem to be working.

The only SSH menu commands working are "Shell", "Firewall logs" and "Update from console".
The menu shows that the OPNSense version is still 24.1

When I try to use "Update from console" the process starts normally, I see the growing list of packaged fetched from repo. But in the end the following happens:

Code: [Select]

[93/93] Fetching py311-h11-0.14.0.pkg: .......... done
Checking integrity... done (0 conflicting)
pkg-static: Cannot delete vital package: opnsense!
pkg-static: If you are sure you want to remove opnsense,
pkg-static: unset the 'vital' flag with: pkg set -v 0 opnsense
ld-elf.so.1: /usr/local/bin/php: Undefined symbol "__libc_start1@FBSD_1.7"
ld-elf.so.1: /usr/local/bin/php: Undefined symbol "__libc_start1@FBSD_1.7"
ld-elf.so.1: /usr/local/bin/php: Undefined symbol "__libc_start1@FBSD_1.7"

So what I my options at the moment?

[franco]

> pkg-static: Cannot delete vital package: opnsense!

IMO this sounds fishy. Try this:

# opnsense-update -p -A 24.7


Cheers,
Franco

[rackenthogg]

> pkg-static: Cannot delete vital package: opnsense!
IMO this sounds fishy. Try this:
# opnsense-update -p -A 24.7

Thank you for fast reply! Here are my results:

Code: [Select]

root@localhost:~ # opnsense-update -p -A 24.7
Updating OPNsense repository catalogue...
pkg-static: Repository OPNsense has a wrong packagesite, need to re-create database
pkg-static: https://pkg.opnsense.org/FreeBSD:13:amd64/24.7/latest/meta.txz: Not Found
repository OPNsense has no meta file, using default settings
pkg-static: https://pkg.opnsense.org/FreeBSD:13:amd64/24.7/latest/packagesite.pkg: Not Found
pkg-static: https://pkg.opnsense.org/FreeBSD:13:amd64/24.7/latest/packagesite.txz: Not Found
Unable to update repository OPNsense
Updating mimugmail repository catalogue...
mimugmail repository is up to date.
Error updating repositories!
root@localhost:~ #


[franco]

Huh, it hasn't even booted the right kernel? oO

# uname -a


Cheers,
Franco

[rackenthogg]

Huh, it hasn't even booted the right kernel? oO
# uname -a

Result:

Code: [Select]

root@localhost:~ # uname -a
FreeBSD localhost.localdomain 14.1-RELEASE-p3 FreeBSD 14.1-RELEASE-p3 stable/24.7-n267778-bb2c86773c1b SMP amd64


[franco]

Ok, and now:

# freebsd-version -u

[rackenthogg]

Ok, and now:

# freebsd-version -u

Code: [Select]

root@localhost:~ # freebsd-version -u
13.2-RELEASE-p11


[franco]

Okay then, the base should have upgraded, but didn't. Let's rerun what's missing:

# opnsense-update -b -A 24.1 -r 24.7

After that try this again:

# opnsense-update -p


Cheers,
Franco

[rackenthogg]

Okay then, the base should have upgraded, but didn't. Let's rerun what's missing:
# opnsense-update -b -A 24.1 -r 24.7

After this nothing works again. No network, no SSH menu, no nothing. I connected display to mainboard's external  VGA port and I was able to see only some quickly flashing error messages about redis database unable to connect and then empty list of network interfaces. Then the screen cleared so I cannot even make the photo of it.
I have my keyboard and display attached directly, so if anyone has further rescue ideas I may work them on my box. I supose I can pause the booting process and access BSD shell somehow.

[doktornotor]

Wouldn't it be a whole lot faster, easier and safer to just do a clean install of the latest version and restore the configuration backup?

[rackenthogg]

Wouldn't it be a whole lot faster, easier and safer to just do a clean install of the latest version and restore the configuration backup?

I will be forced to do so, if I will find no other way.

[franco]

By this time, yes.

What I don't understand is that the rationale was the upgrade was done, but:

1. kernel updated
2. base did not update
3. packages updated

Library function mismatches in final packages. The packages would not easily update without the base being applied. It still wouldn't explain why the base system didn't update. Having only the kernel updated is fine, too. This is the first time I've seen this and I don't understand why fixing the problem ends up breaking it if we already arrived at the problem before...


Cheers,
Franco

[rackenthogg]

This is the first time I've seen this and I don't understand why fixing the problem ends up breaking it if we already arrived at the problem before...
Cheers,
Franco

Well, I suspect this was some unique combination of factors which resulted in this quagmire. It all started when box failed to reboot after an upgrade, so maybe something blocked the upgrade script so it failed to perform all necessary steps.
I did what doktornotor suggested: after the clean install I restored my config.