Hey All -
I'm trying to configure OpnSense to be my router on a remote colocated machine. I'm using out of band management/IPMI to control the server remotely.
I have proxmox installed - it's on a 172.16.5.0/24 network - via Linux Bridge and it's not bound to any physical NIC. I then have a VM with OpnSense - it has 2 virtual NICs - one for the WAN and one for the LAN. There's also a real PCI adapter that I will be passing through to the VM directly that is my WAN connection.
Upon booting the VM, I can ping the 172.16.5.100 (Proxmox) and ping 172.16.5.254 (OpnSense) - so far so good.
Next, I pass through the PCI adapter - so now 3 adapters are passed to the VM - the two internal ephermal adapters - one for LAN, one for WAN and the real PCI ethernet adapter.
Upon booting the VM, I can't ping anything. The VM is functional, the system isn't locked - but it seems like the mapping of the devices or OpnSense simply doesn't like the new adapter appearing and not being configured.
If I shut down the VM, remove the passed through device, restart the VM - I can ping both sides again.
Any ideas?
Thanks
Resolved -
Adding this to my grub file, updating grub and rebooting resolved the issue. Not sure why or how, but thought I'd share in case it helps others:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
I believe if you have an Intel variant the command is slightly different.
IOMMU is required for PCIe passthrough... Well documented.
Whether you had to use PCIs passthrough or not is debatable:
https://forum.opnsense.org/index.php?topic=44159.0 (https://forum.opnsense.org/index.php?topic=44159.0)