[SOLVED] webgui broken after upgrade to 21.1.4

Started by gu6884, March 30, 2021, 10:35:22 PM

Previous topic - Next topic
Hello,

Thank you FRANCO however that does not solve the problem.

# opnsense-revert -r 21.1.3 openssl

Browsers //192.168.66.66:48443
ERR_SSL_PROTOCOL_ERROR

Even with
# configctl webgui restart renew

On the other hand if it can help
# curl -k https://192.168.66.66:48443
curl: (35) error: 1408F10B: SSL routines: ssl3_get_record: wrong version number

Regards,
French mother tongue
Depuis 2017
X7SPA-HF, Intel(R) ATOM(TM) D525, 4Go RAM, 120Go, 2 Lan 24.1.2_1
APU4c, 4Go RAM, 120Go, 4 Lan 24.1.10_8
APU3a, 2Go RAM, 60Go, 3 Lan 24.1.2_1
APU2c, 2Go RAM, 60Go, 3 Lan 23.7.1_3
BIOS A JOUR (v4.19.0.1).

Hello,

I stumbled about the exact same issue when updating from 21.1.3 to 21.1.4 just a few minutes ago. Self-signed certificates (from the system, nothing customized), no LetsEncrypt, neither reboots nor manual webui restarts changed the situation.

Quote
$ curl -k https://fw.domain.tld/
curl: (56) OpenSSL SSL_read: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error, errno 0

The workaround as posted earlier works fine:

Quote
root@fw:/var/log # opnsense-revert -r 21.1.3 openssl
Fetching openssl.txz: .... done
Verifying signature with trusted certificate pkg.opnsense.org.20210104... done
openssl-1.1.1k,1: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        openssl: 1.1.1j_1,1

Number of packages to be installed: 1

The process will require 14 MiB more space.
[1/1] Installing openssl-1.1.1j_1,1...
Extracting openssl-1.1.1j_1,1: 100%
root@fw:/var/log # configctl webgui restart
OK
root@fw:/var/log #

Now it works:
Quote
$ curl -k https://fw.domain.tld/
<!doctype html>
[...]

Regards,
Patrik

Hi @pkernstock,

Thank you and I want to believe that it works for sure.

I followed the instructions well

root@Pare-Feu:/home/henri # opnsense-revert -r 21.1.3 openssl
Fetching openssl.txz: ...... done
Verifying signature with trusted certificate pkg.opnsense.org.20210104... done
openssl-1.1.1j_1,1: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
SunnyValley repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        openssl: 1.1.1j_1,1 [unknown-repository]

Number of packages to be installed: 1

The process will require 14 MiB more space.
[1/1] Installing openssl-1.1.1j_1,1...
Extracting openssl-1.1.1j_1,1: 100%
root@Pare-Feu:/home/henri # configctl webgui restart
OK

browsers FAILED
brave : ERR_SSL_PROTOCOL_ERROR
chrome : ERR_SSL_PROTOCOL_ERROR
edge : ERR_SSL_PROTOCOL_ERROR
firefox : SSL_ERROR_RX_RECORD_TOO_LONG
opera : ERR_SSL_PROTOCOL_ERROR
vivaldi : ERR_SSL_PROTOCOL_ERROR

I restarted but ditto

Regards,
French mother tongue
Depuis 2017
X7SPA-HF, Intel(R) ATOM(TM) D525, 4Go RAM, 120Go, 2 Lan 24.1.2_1
APU4c, 4Go RAM, 120Go, 4 Lan 24.1.10_8
APU3a, 2Go RAM, 60Go, 3 Lan 24.1.2_1
APU2c, 2Go RAM, 60Go, 3 Lan 23.7.1_3
BIOS A JOUR (v4.19.0.1).

@Darkopnsense
Quotefirefox : SSL_ERROR_RX_RECORD_TOO_LONG
Quotecurl: (35) error: 1408F10B: SSL routines: ssl3_get_record: wrong version number
imho there are some problems besides the discussed
can you try with curl -vk?
any clue in /var/log/lighttpd.log?

Hi @fright,

root@Pare-Feu:/home/henri # opnsense-revert -r 21.1.3 openssl
Fetching openssl.txz: ...... done
Verifying signature with trusted certificate pkg.opnsense.org.20210104... done
openssl-1.1.1j_1,1: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
SunnyValley repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        openssl: 1.1.1j_1,1 [unknown-repository]

Number of packages to be installed: 1

The process will require 14 MiB more space.
[1/1] Installing openssl-1.1.1j_1,1...
Extracting openssl-1.1.1j_1,1: 100%
root@Pare-Feu:/home/Stephane # configctl webgui restart
OK
root@Pare-Feu:/home/henri # curl -vk https://192.168.66.66:48443
*   Trying 192.168.66.66:48443...
* Connected to 192.168.66.66 (192.168.66.66) port 48443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /usr/local/etc/ssl/cert.pem
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

I analyze the file lighttpd.log

Regards,
French mother tongue
Depuis 2017
X7SPA-HF, Intel(R) ATOM(TM) D525, 4Go RAM, 120Go, 2 Lan 24.1.2_1
APU4c, 4Go RAM, 120Go, 4 Lan 24.1.10_8
APU3a, 2Go RAM, 60Go, 3 Lan 24.1.2_1
APU2c, 2Go RAM, 60Go, 3 Lan 23.7.1_3
BIOS A JOUR (v4.19.0.1).

Good evening everyone,

After analyzing lighttpd.log, I have reset SENSEI and I am currently accessing the interface with different browsers. And this even after restarting, to be sure.

Regards,
mother tongue French
Depuis 2017
X7SPA-HF, Intel(R) ATOM(TM) D525, 4Go RAM, 120Go, 2 Lan 24.1.2_1
APU4c, 4Go RAM, 120Go, 4 Lan 24.1.10_8
APU3a, 2Go RAM, 60Go, 3 Lan 24.1.2_1
APU2c, 2Go RAM, 60Go, 3 Lan 23.7.1_3
BIOS A JOUR (v4.19.0.1).

This is quite the spectacular breakage somewhere up the food chain:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254643


Cheers,
Franco

PS: does this work too?

# devfs rule apply path crypto hide
# configctl webgui restart

Quote from: franco on April 02, 2021, 07:39:42 PM
PS: does this work too?

# devfs rule apply path crypto hide
# configctl webgui restart

Yes, it does:

Quote
root@iefw01:/var/log # opnsense-revert -r 21.1.4 openssl
Fetching openssl.txz: ... done
Verifying signature with trusted certificate pkg.opnsense.org.20210104... done
openssl-1.1.1j_1,1: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        openssl: 1.1.1k,1

Number of packages to be installed: 1

The process will require 14 MiB more space.
[1/1] Installing openssl-1.1.1k,1...
Extracting openssl-1.1.1k,1: 100%
root@iefw01:/var/log # configctl webgui restart
OK
root@iefw01:/var/log # devfs rule apply path crypto hide
root@iefw01:/var/log # configctl webgui restart
OK
root@iefw01:/var/log #

Then:
Quote$ curl -k https://fw/ | head -n1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2952  100  2952    0     0  38337      0 --:--:-- --:--:-- --:--:-- 38337
<!doctype html>


Welp,  I'm in the same boat.  WebGUI is unavailable after upgrade.

Instead of rolling back the one package as mentioned here, I rolled back the ZFS boot environment. (I always create a new ZFS boot environment before upgrading as a 'just in case' situation)

I'm curious to know what the actual problem is, and I'll wait until the issue is fixed in an official release, then I'll try upgrading again.


I've tried "crypto hide" solution and it works for me.
As my openssl package was already reverted, I did
# devfs rule apply path crypto hide
# opnsense-revert openssl
# configctl webgui restart

WebGUI is still available.


So the simplest workaround for now is:
# devfs rule apply path crypto hide
# configctl webgui restart

These two commands should restore the WebGUI access.

Thanks all so far. The following package should work:

# pkg add -f https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/misc/openssl-1.1.1k,1.txz

We will do a hotfix, but not today as there is nobody in the office to verify the build. So that will probably be tomorrow.

What this means is that /dev/crypto OpenSSL engine support is going to be disabled due to broken patches added in 1.1.1k. I'll leave you to look into who and why...

LibreSSL removed /dev/crytpo support a long time ago, but we still have System: Settings: Miscellaneous
"Use /dev/crypto" non-default settings which broke this for involved users. We ask you to switch this option off now as it is likely being removed from 21.7 to avoid further problems.


Cheers,
Franco

Quote from: franco on April 05, 2021, 01:41:21 PM
Thanks all so far. The following package should work:

# pkg add -f https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/misc/openssl-1.1.1k,1.txz
Thanks!

Tested by
# devfs rule apply path crypto unhide
# pkg add -f https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/misc/openssl-1.1.1k,1.txz
# configctl webgui restart


WebGUI is available!

Thanks a lot for testing @karlson2k

Hotfix went out this morning, update should show up for anyone who still is on the original 21.1.4. Mostly this is to avoid users from below 21.1.4 to trip over the same thing.


Cheers,
Franco