Messages

I recently set up a new hypervisor using vm-bhyve. I ran into an issue passing through an Intel dual port 82576 NIC and a Mellanox ConnectX-4 Lx MCX4121A-XCAT. I searched the web but did not find anyone discussing this issue.

Following the vm-bhyve man page, I reserved the devices by adding the device ID as shown with 

Code Select Expand

pciconf -lv to the pptdevs variable in /boot/loader.conf.

Mellanox: 1/0/0 and 1/0/1
Intel NIC: 5/0/0 and 5/0/1

Then I rebooted and added the devices to the vm config file.

Code Select Expand

passthru0="5/0/0=8:0"
passthru1="5/0/1=8:1"
passthru2="1/0/0=9:0"
passthru3="1/0/1=9:1"

The VM would boot, but the nics would show in the VM intermittently and did not work most of the time.

To fix this I went back to /boot/loader.conf and reserved all possible functions (0-7) in the pptdevs variable, not just the ones displayed in pciconf -lv. This fixed the problem.

Code Select Expand

pptdevs="5/0/0 5/0/1 5/0/2 5/0/3 5/0/4 5/0/5 5/0/6 5/0/7 1/0/0 1/0/1 1/0/2 1/0/3 1/0/4 1/0/5 1/0/6 1/0/7"

I hope this helps someone trying to virtualize opnsense using bhyve.

I went ahead and purchased the AMD 5825U version with 4 i226. It has been perfectly stable with proxmox along side other vms. I had issues with intel 5105 regardless of tweaks. I recommend paying extra for the AMD even if just for piece of mind.

This is a common issue with those units. There isn't any hardware swapping or configuration tweaking you can do to fix it.

My suggestion right now is to upgrade the host kernel to the latest version 5.19 and report back your results. Maybe even try using proxmox just to work alongside the efforts of others.

Kernel upgrade thread: https://forum.proxmox.com/threads/opt-in-linux-5-19-kernel-for-proxmox-ve-7-x-available.115090/

Main thread tracking this issue: https://forum.proxmox.com/threads/vm-freezes-irregularly.111494/