Anyone running OPNsense in bhyve? Reboot hangs

[Patrick M. Hausen]

Hi all,

I just installed an OPNsense 23.7 in bhyve in a TrueNAS CORE 13.0-U5.3. 2 network interfaces via PCIe passthrough and generally everything is working great.

Only minor problem: when I try to soft reboot the VM, the filesystem buffers are flushed and the last message I see is "ix1 changed state to DOWN" - then the guest system hangs until powered off from the host and powered on again. Identical for soft power off. The system shuts down but the virtual/emulated power off does not happen.

I vaguely remember that occasionally people have this problem running on real hardware too and that you can twiddle the reset method via loader.conf somehow (ACPI vs. ... what?)

All hints greatly appreciated. I'm running this on a tiny Supermicro system. That would make a great chimera for travel - TrueNAS and OPNsense all in one box.

[Patrick M. Hausen]

OK, the shutdown/poweroff could be fixed by

sysctl hw.efi.poweroff=0

Reboot still hanging.

[Patrick M. Hausen]

Switching from PCIe passthrough to bridge interfaces (in TrueNAS) and VirtIO (in OPNsense) fixes the problem and seems to work well.

No tunables needed at all and with 4 cores assigned to the VM I get near gigabit speed.