Running OpnSense in a VM, works fine until passing through Network Adapter

[theprez1980]

Hey All -

I'm trying to configure OpnSense to be my router on a remote colocated machine.  I'm using out of band management/IPMI to control the server remotely.

I have proxmox installed - it's on a 172.16.5.0/24 network - via Linux Bridge and it's not bound to any physical NIC.  I then have a VM with OpnSense - it has 2 virtual NICs - one for the WAN and one for the LAN.  There's also a real PCI adapter that I will be passing through to the VM directly that is my WAN connection.

Upon booting the VM, I can ping the 172.16.5.100 (Proxmox) and ping 172.16.5.254 (OpnSense) - so far so good.

Next, I pass through the PCI adapter - so now 3 adapters are passed to the VM - the two internal ephermal adapters - one for LAN, one for WAN and the real PCI ethernet adapter.

Upon booting the VM, I can't ping anything.  The VM is functional, the system isn't locked - but it seems like the mapping of the devices or OpnSense simply doesn't like the new adapter appearing and not being configured.

If I shut down the VM, remove the passed through device, restart the VM - I can ping both sides again.

Any ideas?

Thanks

[theprez1980]

Resolved -

Adding this to my grub file, updating grub and rebooting resolved the issue.  Not sure why or how, but thought I'd share in case it helps others:

Code Select Expand

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
I believe if you have an Intel variant the command is slightly different.

[EricPerl]

IOMMU is required for PCIe passthrough... Well documented.

Whether you had to use PCIs passthrough or not is debatable:
https://forum.opnsense.org/index.php?topic=44159.0