Messages

[Don't use this in production. Do try this in your lab so we could exercise this code path and work out the kinks ;)]

Greetings. Following in the footsteps of slipperyduck &co, I managed to convert my OPNsense router into a virtualization host, and I'd like to register my positive signal. One could debate the merits of running OPNsense on bare metal to host virtual machines vs running OPNsense itself as a virtual machine alongside other virtual machines vs using separate physical machines. Suffice it to say that there may be security implications of using OPNsense as a virtualization host that are unknown as of yet because this is an uncommon setup.

Don't use this in production. Do try this in your lab so we could exercise this code path and work out the kinks ;)

Anyway, it all begins with the installation of the popular vm-bhyve system and required GRUB loader from the FreeBSD repository:

Code Select Expand

pkg lock -y pkg
sed -i '' 's/enabled: no/enabled: yes/' /usr/local/etc/pkg/repos/FreeBSD.conf
pkg install -y vm-bhyve grub2-bhyve
sed -i '' 's/enabled: yes/enabled: no/' /usr/local/etc/pkg/repos/FreeBSD.conf
pkg unlock -y pkg

Next, we prepare vm-bhyve (I'm using ZFS here):

Code Select Expand

zfs create zroot/vm
sysrc vm_enable="YES"
sysrc vm_dir="zfs:zroot/vm"
vm init
cp /usr/local/share/examples/vm-bhyve/* /zroot/vm/.templates/

Here, we return to OPNsense's web UI to configure the network bridge. To make things easy, we could create a bridge for the LAN and then (later) use the same bridge for the VMs. My bridge is called bridge0 (the device name in the Device field).

In addition, I recommend doing packet filtering on the bridge interface instead of the individual ports, so we could connect any number of VMs without reconfiguring the firewall. In System > Settings > Tunables, set net.link.bridge.pfil_member=0 and net.link.bridge.pfil_bridge=1 (sysctl works too, but the settings wouldn't survive reboot).

Back to the command line, we tell vm-bhyve about our bridge:

Code Select Expand

vm switch create -t manual -b bridge0 public

Now let's choose an OS for our VM. I'm going lightweight with Alpine:

Code Select Expand

vm iso https://dl-cdn.alpinelinux.org/alpine/v3.17/releases/x86_64/alpine-standard-3.17.2-x86_64.iso

The Alpine template requires a little tweak:

Code Select Expand

sed -i '' 's/vanilla/lts/g' /zroot/vm/.templates/alpine.conf

And now, the moment of truth...

Code Select Expand

vm create -t alpine -s 1G myfirstvm
vm install -f myfirstvm alpine-standard-3.17.2-x86_64.iso

We should now be able to install the OS. I don't know how to exit the nmdm console during installation (~ ^D did not work for me here) short of disconnecting the SSH session; maybe y'all could help me out ;)

After installation, we should be able to use the vm start myfirstvm and vm stop myfirstvm commands to manage the VM. Running vm console myfirstvm connects to the console (and here, ~ ^D did work to disconnect).

If all works well, we could make this VM start upon host boot:

Code Select Expand

sysrc vm_list="myfirstvm"

Protip: The bhyvectl tool could be used to manage the VMs directly if vm-bhyve ran into issues. If bhyvectl were used to stop a VM, the /zroot/vm/<name>/run.lock file should be removed to avoid confusing vm-bhyve.

Now all we need is a slick web UI ;)