OPNsense Forum

Archive => 20.1 Legacy Series => Topic started by: moware on February 14, 2020, 04:14:20 pm

Title: Web UI gone after upgrade from 19.7.10 to 20.1(.1?)
Post by: moware on February 14, 2020, 04:14:20 pm
I just upgraded my Decisio appliance (OPNsense GHz small) from 19.7.10 to 20.1 via the Web UI. This is what happened:

1. After downloading everything, the web UI told me to wait for a reboot.
2. After half an hour, the appliance was still unavailable. This had already happened during the 19.1 -> 19.7 upgrade (see https://forum.opnsense.org/index.php?topic=13749.msg63309 (https://forum.opnsense.org/index.php?topic=13749.msg63309)), so I didn't worry and power cycled the device.
3. The device was quickly back online, with some services working (NAT, WAN failover), and others not working (OpenVPN, Web UI). Yes, this means that I cannot access the web UI any more!

nmap shows that no ports are open on the LAN interface of the device. I tried another power cycle, but it didn't help.

I just ordered a null-modem cable and a USB-serial adapter to see if I can debug this issue via the serial console; both should arrive next week.

Any other hints on what I can try in the meantime?
Title: Re: Web UI gone after upgrade from 19.7.10 to 20.1(.1?)
Post by: moware on February 17, 2020, 02:31:43 pm
Using the serial console access, I was able to fix the issue. If anyone ever encounters a similar issue, this is what worked for me.

The console showed the following error during startup:

Code: [Select]
Configuring firewall.......done.
Starting PFLOG...done.
Syncing OpenVPN settings...
Fatal error: Uncaught Error: Call to undefined function openssl_x509_parse() in /usr/local/etc/inc/certs.inc:391
Stack trace:
#0 /usr/local/etc/inc/certs.inc(727): cert_get_subject_array('LS0tLS1CRUdJTiB...')
#1 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(759): cert_get_cn('LS0tLS1CRUdJTiB...')
#2 /usr/local/etc/inc/plugins.inc.d/openvpn.inc(1228): openvpn_reconfigure('server', Array, false)
#3 /usr/local/etc/inc/plugins.inc(243): openvpn_configure_do(true)
#4 /usr/local/etc/rc.bootup(114): plugins_configure('vpn', true)
#5 {main}
  thrown in /usr/local/etc/inc/certs.inc on line 391
Enter full pathname of shell or RETURN for /bin/sh:
I pressed return to enter the shell and ran opnsense-update, which yielded the following output:

Code: [Select]
The following 5 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
        openssl-1.1.1d,1

New packages to be INSTALLED:
        openssl102: 1.0.2u

Installed packages to be REINSTALLED:
        python37-3.7.6 (direct dependency changed: openssl102)
        py37-cryptography-2.6.1 (direct dependency changed: openssl102)
        cyrus-sasl-2.1.27_1 (direct dependency changed: openssl102)
Now the reason for the problem was obvious: The 20.1 version of OpenSSL was running together with the 19.7 version of OpenVPN. Apparently, the upgrade to 20.1 had only been half done, leading to mutually incompatible packages.

The fix for this was easy: Restore the system to a working 19.7 configuration with opnsense-update, then upgrade to 20.1 using the console boot menu.

Lessons learned:
Title: Re: Web UI gone after upgrade from 19.7.10 to 20.1(.1?)
Post by: franco on February 17, 2020, 02:39:35 pm
Quote from: moware on February 17, 2020, 02:31:43 pm
Lessons learned:
  • Major updates can take a long time on slow devices. Plan for a long downtime and don't be impatient!
  • The OPNsense documentation recommends to perform major upgrades via VGA display or serial. Heed that advice!
  • The Deciso appliances ship with a USB-to-serial null modem cable already included, which is awesome!

This is top notch advice. :)

Glad you could fix it.


Cheers,
Franco