"Danger. Unexpected error, check log for details" during 25.7.6 upgrade

Started by jonm, October 23, 2025, 05:18:07 PM

Previous topic - Next topic
I just got this rather scary message during the update to 25.7.6. The update appeared to stop.

I couldn't log in at the console, I got this error:

Password:
sh: /usr/local/libexec/opnsense-auth: not found
Login incorrect

The GUI then gave a 404 error.

After a couple of minutes it sprang back into life.

I've never seen this behaviour before - is it expected?

The update appears to have now completed successfully, as far as I can tell.

We're investigating these reports which seem to be more than usual... which are likely due to the new package manager update behaviour.

It looks like the core package was deinstalled, some packages updated but ultimately failed and the core package not put back. I can't say more for lack of evidence, but if that's the case I know what to fix.

In your case the system is probably not in a recoverable state, but a config import from install media and reinstall will bring you back.


Cheers,
Franco

Thanks Franco. I can provide logs if they might be of help. The system seems to be ok now.

Can you check if

# opnsense-update -g

yields anything useful anymore? This is where the last update log is stored for debugging, but it's only the last and overwritten on the next update attempt.


Thanks,
Franco


Well, it spreads out the deinstall/install of the core package, but it has all the relevant bits:

[8/136] Deinstalling opnsense-25.7.3_7...
Stopping configd...done
Resetting root shell
Updating /etc/shells
Unhooking from /etc/rc
Unhooking from /etc/rc.shutdown
[8/136] Deleting files for opnsense-25.7.3_7: .......... done

[...]

[136/136] Installing opnsense-25.7.6...
[136/136] Extracting opnsense-25.7.6: .......... done
Updating /etc/shells
Registering root shell
Hooking into /etc/rc
Hooking into /etc/rc.shutdown
Starting configd.
>>> Invoking update script 'refresh.sh'
Migrated OPNsense\IDS\IDS from 1.1.0 to 1.1.1
Migrated OPNsense\Wireguard\Server from 1.0.0 to 1.0.1
Flushing all caches...done.
Writing firmware settings: FreeBSD OPNsense
Writing trust files...done.
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
certctl: No changes to trust store were made.
Writing trust bundles...done.
Configuring login behaviour...done.
Configuring cron...done.
Configuring system logging...done.

You can see it installed the core files and did its maintenance tasks but after reboot the core files (or at least some) were gone from the file system?

We can use an additional explicit sync before reboot but it feels fishy.


Cheers,
Franco

Quote from: jonm on October 23, 2025, 05:18:07 PMI just got this rather scary message during the update to 25.7.6. The update appeared to stop.

I couldn't log in at the console, I got this error:

Password:
sh: /usr/local/libexec/opnsense-auth: not found
Login incorrect

The GUI then gave a 404 error.

After a couple of minutes it sprang back into life.

I've never seen this behaviour before - is it expected?

The update appears to have now completed successfully, as far as I can tell.

I had the same "Danger. Unexpected error, check log for details" error but when I refreshed the OPNsense main dashboard page, version updated to 25.7.6 and a check for updates came back with "There are no updates available on the selected mirror".

I am reluctant to reboot in case I'm left with a broken OPNsense.

Is there anything I can check to ensure a reboot will succeed?

Reading this again:

QuoteI couldn't log in at the console, I got this error:

Password:
sh: /usr/local/libexec/opnsense-auth: not found
Login incorrect

The GUI then gave a 404 error.

After a couple of minutes it sprang back into life.

So it started working again when the update finished like it said? In that case it's not really fatal but expected as long as opnsense core package is not physically installed. It's an annoying situation, but certainly not an issue like a reboot not working.


Cheers,
Franco

I've also experienced this error updating today updating from 25.7.4.  Error log linked below.

https://pastebin.com/HvPkPHLj

In my case opnsense is running as a VM and I rolled back to a pre-update snapshot.  If you need me to attempt the update again and perform any diagnostics before rolling back let me know.

Edit: I tried again from a clean reboot and exact same issue.

The log stops but nothing that would indicate the issue.

Can you clarify:

> Edit: I tried again from a clean reboot and exact same issue.

?

The system had been online for a while prior to the update so I rebooted and attempted the update again. Same problem, logs ended at the same place. I can run the update again and provide any additional logs if you tell me which ones to pull.

Tried again because I'm stubborn and I was hoping to pull the contents of /var/log after the update. Once the update fails I am no longer able to login via ssh or the console:

> sh: /usr/local/libexec/opnsense-auth: not found

root@fw:/var/log/pkg # cat pkg_20251024.log
<13>1 2025-10-24T08:35:18-04:00 fw.lan pkg-static 81937 - [meta sequenceId="1"] libcbor upgraded: 0.12.0_2 -> 0.13.0
<13>1 2025-10-24T08:35:20-04:00 fw.lan pkg-static 81937 - [meta sequenceId="2"] libunistring upgraded: 1.3 -> 1.4.1
<13>1 2025-10-24T08:35:26-04:00 fw.lan pkg-static 81937 - [meta sequenceId="3"] crowdsec-1.7.0 deinstalled
<13>1 2025-10-24T08:35:31-04:00 fw.lan pkg-static 81937 - [meta sequenceId="4"] crowdsec-firewall-bouncer upgraded: 0.0.32_5 -> 0.0.32_7
<13>1 2025-10-24T08:35:43-04:00 fw.lan pkg-static 81937 - [meta sequenceId="5"] crowdsec-1.7.0_2 installed
<13>1 2025-10-24T08:35:44-04:00 fw.lan pkg-static 81937 - [meta sequenceId="6"] git-2.51.0 deinstalled
<13>1 2025-10-24T08:35:47-04:00 fw.lan pkg-static 81937 - [meta sequenceId="7"] opnsense-25.7.4 deinstalled
<13>1 2025-10-24T08:35:49-04:00 fw.lan pkg-static 81937 - [meta sequenceId="8"] dnsmasq reinstalled: 2.91_1,1 -> 2.91_1,1
<13>1 2025-10-24T08:36:15-04:00 fw.lan pkg-static 81937 - [meta sequenceId="9"] php83-phpseclib upgraded: 3.0.46 -> 3.0.47
<13>1 2025-10-24T08:36:17-04:00 fw.lan pkg-static 81937 - [meta sequenceId="10"] kea-3.0.1_1 deinstalled
<13>1 2025-10-24T08:36:18-04:00 fw.lan pkg-static 81937 - [meta sequenceId="11"] ntp-4.2.8p18_4 deinstalled
<13>1 2025-10-24T08:36:18-04:00 fw.lan pkg-static 81937 - [meta sequenceId="12"] openssh-portable-10.0.p1_2,1 deinstalled
<13>1 2025-10-24T08:36:19-04:00 fw.lan pkg-static 81937 - [meta sequenceId="13"] openvpn-2.6.15 deinstalled
<13>1 2025-10-24T08:36:19-04:00 fw.lan pkg-static 81937 - [meta sequenceId="14"] opnsense-update-25.7.3 deinstalled
<13>1 2025-10-24T08:36:20-04:00 fw.lan pkg-static 81937 - [meta sequenceId="15"] os-ddclient-1.27_4 deinstalled
<13>1 2025-10-24T08:36:20-04:00 fw.lan pkg-static 81937 - [meta sequenceId="16"] php83-ldap-8.3.26 deinstalled
<13>1 2025-10-24T08:36:26-04:00 fw.lan pkg-static 81937 - [meta sequenceId="17"] openldap26-client-2.6.10 deinstalled
<13>1 2025-10-24T08:36:27-04:00 fw.lan pkg-static 81937 - [meta sequenceId="18"] cyrus-sasl-gssapi-2.1.28 deinstalled
<13>1 2025-10-24T08:36:27-04:00 fw.lan pkg-static 81937 - [meta sequenceId="19"] cyrus-sasl-2.1.28_5 deinstalled
<13>1 2025-10-24T08:36:29-04:00 fw.lan pkg-static 81937 - [meta sequenceId="20"] krb5-1.22.1 deinstalled
<13>1 2025-10-24T08:36:30-04:00 fw.lan pkg-static 81937 - [meta sequenceId="21"] py311-anyio-4.10.0 deinstalled
<13>1 2025-10-24T08:36:31-04:00 fw.lan pkg-static 81937 - [meta sequenceId="22"] py311-boto3-1.40.21 deinstalled
<13>1 2025-10-24T08:36:31-04:00 fw.lan pkg-static 81937 - [meta sequenceId="23"] py311-cryptography-44.0.3_3,1 deinstalled
<13>1 2025-10-24T08:36:32-04:00 fw.lan pkg-static 81937 - [meta sequenceId="24"] py311-markupsafe-3.0.2 deinstalled
<13>1 2025-10-24T08:36:32-04:00 fw.lan pkg-static 81937 - [meta sequenceId="25"] py311-numexpr-2.11.0 deinstalled
<13>1 2025-10-24T08:36:32-04:00 fw.lan pkg-static 81937 - [meta sequenceId="26"] py311-pycparser-2.22 deinstalled
<13>1 2025-10-24T08:36:33-04:00 fw.lan pkg-static 81937 - [meta sequenceId="27"] py311-s3transfer-0.13.1 deinstalled
<13>1 2025-10-24T08:36:37-04:00 fw.lan pkg-static 81937 - [meta sequenceId="28"] py311-botocore-1.40.21 deinstalled
<13>1 2025-10-24T08:36:38-04:00 fw.lan pkg-static 81937 - [meta sequenceId="29"] py311-sqlite3-3.11.13_11 deinstalled
<13>1 2025-10-24T08:36:38-04:00 fw.lan pkg-static 81937 - [meta sequenceId="30"] py311-urllib3-1.26.20,1 deinstalled
<13>1 2025-10-24T08:36:38-04:00 fw.lan pkg-static 81937 - [meta sequenceId="31"] qemu-guest-agent-10.1.0 deinstalled
<13>1 2025-10-24T08:36:39-04:00 fw.lan pkg-static 81937 - [meta sequenceId="32"] rrdtool-1.9.0_1 deinstalled
<13>1 2025-10-24T08:36:39-04:00 fw.lan pkg-static 81937 - [meta sequenceId="33"] rubygem-rexml-3.4.2 deinstalled
<13>1 2025-10-24T08:36:41-04:00 fw.lan pkg-static 81937 - [meta sequenceId="34"] ruby33-gems-3.7.1 deinstalled
<13>1 2025-10-24T08:36:47-04:00 fw.lan pkg-static 81937 - [meta sequenceId="35"] ruby-3.3.9,1 deinstalled
<13>1 2025-10-24T08:36:50-04:00 fw.lan pkg-static 81937 - [meta sequenceId="36"] strongswan-6.0.1 deinstalled
<13>1 2025-10-24T08:36:51-04:00 fw.lan pkg-static 81937 - [meta sequenceId="37"] sudo-1.9.17p2 deinstalled
<13>1 2025-10-24T08:36:52-04:00 fw.lan pkg-static 81937 - [meta sequenceId="38"] suricata-7.0.12 deinstalled
<13>1 2025-10-24T08:36:58-04:00 fw.lan pkg-static 81937 - [meta sequenceId="39"] nss upgraded: 3.116 -> 3.117
<13>1 2025-10-24T08:36:58-04:00 fw.lan pkg-static 81937 - [meta sequenceId="40"] py311-pyyaml-6.0.1_1 deinstalled
<13>1 2025-10-24T08:36:59-04:00 fw.lan pkg-static 81937 - [meta sequenceId="41"] syslog-ng-4.8.2_4 deinstalled
<13>1 2025-10-24T08:37:00-04:00 fw.lan pkg-static 81937 - [meta sequenceId="42"] curl-8.15.0 deinstalled
<13>1 2025-10-24T08:37:00-04:00 fw.lan pkg-static 81937 - [meta sequenceId="43"] glib-2.84.1_3,2 deinstalled
<13>1 2025-10-24T08:37:13-04:00 fw.lan pkg-static 81937 - [meta sequenceId="44"] pcre2 upgraded: 10.45_1 -> 10.46
<13>1 2025-10-24T08:37:13-04:00 fw.lan pkg-static 81937 - [meta sequenceId="45"] tailscale-1.88.1 deinstalled
<13>1 2025-10-24T08:37:14-04:00 fw.lan pkg-static 81937 - [meta sequenceId="46"] ca_root_nss upgraded: 3.115_2 -> 3.115_3
<13>1 2025-10-24T08:37:15-04:00 fw.lan pkg-static 81937 - [meta sequenceId="47"] tailscale-1.88.3_2 installed
<13>1 2025-10-24T08:37:15-04:00 fw.lan pkg-static 81937 - [meta sequenceId="48"] unbound-1.24.0 deinstalled
<13>1 2025-10-24T08:37:16-04:00 fw.lan pkg-static 81937 - [meta sequenceId="49"] expat upgraded: 2.7.1 -> 2.7.3
<13>1 2025-10-24T08:37:17-04:00 fw.lan pkg-static 81937 - [meta sequenceId="50"] libnghttp2 upgraded: 1.67.0 -> 1.67.1
<13>1 2025-10-24T08:37:19-04:00 fw.lan pkg-static 81937 - [meta sequenceId="51"] python311-3.11.13_1 deinstalled
<13>1 2025-10-24T08:37:19-04:00 fw.lan pkg-static 81937 - [meta sequenceId="52"] wpa_supplicant-2.11_5 deinstalled

Quote from: RutgerDiehard on October 24, 2025, 10:25:37 AM
Quote from: jonm on October 23, 2025, 05:18:07 PMI just got this rather scary message during the update to 25.7.6. The update appeared to stop.

I couldn't log in at the console, I got this error:

Password:
sh: /usr/local/libexec/opnsense-auth: not found
Login incorrect

The GUI then gave a 404 error.

After a couple of minutes it sprang back into life.

I've never seen this behaviour before - is it expected?

The update appears to have now completed successfully, as far as I can tell.

I had the same "Danger. Unexpected error, check log for details" error but when I refreshed the OPNsense main dashboard page, version updated to 25.7.6 and a check for updates came back with "There are no updates available on the selected mirror".

I am reluctant to reboot in case I'm left with a broken OPNsense.

Is there anything I can check to ensure a reboot will succeed?


Same here. I received the error message, but after refreshing, everything is running fine.

Have not rebooted yet, due to the same concern.
[HW]
Protectli VP2420
16GB RAM
240 GB SSD

[Versions]
OPNsense 25.7.6-amd64
FreeBSD 14.3-RELEASE-p4
OpenSSL 3.0.18

[Feature set]
Unbound DNS
Kea DHCPv4
Suricata IPS
Wireguard Client VPN

October 24, 2025, 06:10:09 PM #14 Last Edit: October 24, 2025, 06:45:26 PM by OIT Ad Mins Of The Day
Unfortunately, I can chime in here...

I guess that's not a good sign?



Update: After reloading the web page, the web frontend shows again - though not sure how to proceed best...

Not sure if I even have ssh access to the box... Looks like management through the UI worked just too well for too long so far...

Also have not rebooted so far, fearing the box won't boot up again...

Maybe pull this update until the cause for the issues has been found?