Installation stuck as bhyve guest (fails to mount root)

[soupdiver]

I try to install opnsense as a bhyve guest. So I can use my Intel NUC for more than "jsut" opnsense.

Code Select Expand


iohyve create opn 8G
iohyve set opn ram=2G
iohyve install opn opn.img


This is how I try to install. When attaching to the console I see:

Code Select Expand


Booting...
KDB: debugger backends: ddb
KDB: current backend: ddb
---<<BOOT>>---
Copyright (c) 2013-2019 The HardenedBSD Project.
Copyright (c) 1992-2019 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.1-RELEASE-p7-HBSD #0  427d53bc125(stable/20.7)-dirty: Sun Jul 26 05:51:42 CEST 2020
    root@sensey64:/usr/obj/usr/src/amd64.amd64/sys/SMP amd64
FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on LLVM 8.0.1)
VT: init without driver.
HardenedBSD: initialize and check features (__HardenedBSD_version 1200059 __FreeBSD_version 1201000).
CPU: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz (2712.19-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x806ea  Family=0x6  Model=0x8e  Stepping=10
  Features=0x9f83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS,HTT,PBE>
  Features2=0xfeda7a17<SSE3,PCLMULQDQ,DTES64,DS_CPL,SSSE3,SDBG,FMA,CX16,xTPR,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
  AMD Features=0x24100800<SYSCALL,NX,Page1GB,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x40729<FSGSBASE,BMI1,AVX2,BMI2,ERMS,INVPCID,RDSEED>
  Structured Extended Features3=0x400<MD_CLEAR>
  XSAVE Features=0x1<XSAVEOPT>
  TSC: P-state invariant
Hypervisor: Origin = "bhyve bhyve "
real memory  = 536870912 (512 MB)
avail memory = 478711808 (456 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <BHYVE  BVMADT  >
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-31 on motherboard
Timecounter "TSC-low" frequency 1356094593 Hz quality 1000
wlan: mac acl policy registered
random: entropy device external interface
kbd1 at kbdmux0
module_register_init: MOD_LOAD (vesa, 0xffffffff812507c0, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
000.000050 [4335] netmap_init               netmap: loaded module
[ath_hal] loaded
nexus0
cryptosoft0: <software crypto> on motherboard
acpi0: <BHYVE BVXSDT> on motherboard
acpi0: Power Button (fixed)
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 16777216 Hz quality 950
Event timer "HPET" frequency 16777216 Hz quality 550
Event timer "HPET1" frequency 16777216 Hz quality 450
Event timer "HPET2" frequency 16777216 Hz quality 450
Event timer "HPET3" frequency 16777216 Hz quality 450
Event timer "HPET4" frequency 16777216 Hz quality 450
Event timer "HPET5" frequency 16777216 Hz quality 450
Event timer "HPET6" frequency 16777216 Hz quality 450
Event timer "HPET7" frequency 16777216 Hz quality 450
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: could not evaluate _ADR - AE_NOT_FOUND
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel ICH8 AHCI SATA controller> mem 0xc0000000-0xc00003ff irq 16 at device 2.0 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
virtio_pci0: <VirtIO PCI Network adapter> port 0x2000-0x201f mem 0xc0002000-0xc0003fff irq 17 at device 3.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci0
vtnet0: Ethernet address: 00:a0:98:d1:df:cb
vtnet0: netmap queues/slots: TX 1/1024, RX 1/512
000.000162 [ 503] vtnet_netmap_attach       vtnet attached txq=1, txd=1024 rxq=1, rxd=512
ahci1: <Intel ICH8 AHCI SATA controller> mem 0xc0004000-0xc00043ff irq 18 at device 4.0 on pci0
ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich6: <AHCI channel> at channel 0 on ahci1
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
driver bug: Unable to set devclass (class: atkbdc devname: (unknown))
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model Generic PS/2 mouse, device ID 0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
vga0: <Generic ISA VGA> at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid PNP0900 on isa0
Timecounters tick every 10.000 msec
usb_needs_explore_all: no devclass
cd0 at ahcich6 bus 0 scbus1 target 0 lun 0
cd0: <BHYVE BHYVE DVD-ROM 001> Removable CD-ROM SCSI device
cd0: Serial Number BHYVE-30A4-077D-85D6
cd0: 600.000MB/s transfers (SATA 3.x, WDMA2, ATAPI 12bytes, PIO 8192bytes)
cd0: 1782MB (912672 2048 byte sectors)
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <BHYVE SATA DISK 001> ACS-2 ATA SATA 3.x device
ada0: Serial Number BHYVE-B887-3F3E-2039
ada0: 600.000MB/s transfers (SATA 3.x, WDMA2, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 8192MB (16777216 512 byte sectors)

Loader variables:

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:zroot/ROOT/default
        cd9660:/dev/cd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input


So it seems that for some reason my root disk ada0 can not be mounted?
I already tried https://github.com/opnsense/tools/issues/28 but also no luck. Not even sure if this is related but...

I downloaded the amd64 console image

Any ideas?