17.1 on ESXi boots impossibly slow (video)

[CloudHoppingFlowerChild]

Hello.
Thanks to everyone here for helping build this community and opnsense.

I'm trying to upgrade from 16.7.14_2 to 17.1. My router is virtualized running on a VMware ESXi 6.5 host. The update process looks very smooth, it reboots and everything loads quickly/normally to a point. After that point, the display updates line, by bloody line. Like I'm watching some text load up in a console over a 2400 baud connection.

This begins with logging in and starting the installation
https://youtu.be/8UM0aBXRSkA

This skips to where the installation has completed and the system reboots. The problem starts 10 seconds after:
https://youtu.be/8UM0aBXRSkA?t=1m10s

This is a video capture I made of the installation process. The problem starts 1:19 into the video. I've let it sit for  a couple hours but it didn't finish loading. During this time the CPU usage goes to 100% and memory stays flat at 2.15GB/4GB. I've tried tweaking the VM settings like changing the Guest OS setting, increasing and decreasing video memory, disabling vmware tools checks, but it always has the same problem during the initial boot following the console initiated update.

Thanks for any input.

[shezzski]

Hi CloudHoppingFlowerChild,

I may not be able to assist you directly, but I have been able to perform the update though I am using ESXi 6.0 Build 4600944.

My OPNsense VM has 348MB of RAM, 1 vCPU, 8GB HDD and 1.18MB of video memory.  Happy to run through all my settings if you think it would be useful.

[mr.rosh]

as a matter of isolation, deploy a new vm [latest vm hardware level] and perform fresh installation on 17.1 [ensure you select OS type as BSD equivalent]

deploy vmware tools. not from the vmware vsphere. but from the opnsense updates/tools section.
reboot and see outcomes.


if it works ok, deploy the config file from previous old/opnsense and into new one.

[weust]

I run Hyper-V at home, and since FreeBSD doesn't have a graphics card driver, this can happen to me too.
What I do then I edit the loader.conf (iirc, that file) to set it to tekst only console.

But, since you're saying it takes several hours your problem might be different from mine.

[CloudHoppingFlowerChild]

I've worked it out, for the most part.
The problem, as far as I can understand it, has to do with the firmware type (BIOS vs EFI):


When set to BIOS the console works normally. I'm able to do a clean install when set to BIOS because the console keeps updating and I can see what I'm doing. I set my original OPNsense VM to use BIOS (was EFI) and I was then able to execute a successful update from the console. On a new VM, a fresh install was no problem when set to BIOS. After the installation was complete and and OPNsense was running, I can switch to EFI and I'm able to reboot and OPNsense works, but the console stops updating/freezes at this point:


Which is what shows in my upgrade video immediately before the console starts redrawing the screen line by line. That's as deep as my analysis can go; a problem between the VMware EFI and the screen draw functions of FreeBSD 11 or whatever and it plays nice when set to BIOS mode.

The vt/primary/secondary console settings under System>Settings>Administration have no effect on the console working with an EFI based VM.

[franco]

What do other VMware/ESXi users use? Does 16.7.x work better in EFI mode (FreeBSD 10.3)?

I did some tests for the 16.1 release in VMware Fusion, they seemed fine, but it was another FreeBSD entirely (10.2).


Cheers,
Franco

[franco]

We have a winner:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214518
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203682

The proposed commit enables EARLY_AP_STARTUP in the kernel, which has caused different side effects on 12-CURRENT. Not sure this will make it to OPNsense before 17.7.


Cheers,
Franco