Virtualized opnsense: SR-IOV VF support questions

[arglebargle]

Is anyone passing an SR-IOV VF to an opnsense guest successfully? I've been researching nic drivers for a few days now and there's not a lot of info online about what actually works. This is really a FreeBSD nic driver issue, I'm just hoping someone here has done this and can relay their experience.

I've got a few different cards in the mail from eBay sellers so I can test them myself but I'd really appreciate hearing about any experience that anyone else has had with this.

I'm targeting 10GbE adapters for my build -- I've already tested the mlx4 driver with CX-2 and CX-3 cards that I have on hand and I've had some success using VFs with the mlx4 driver in FreeBSD 12 but FreeBSD 11 is a complete no-go.

I understand Chelsio and Intel are the most supported, has anyone used these with SR-IOV?

[julien_]

Looks like freebsd does not support it.
https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/supported-freebsd-virtual-machines-on-hyper-v

[tokade]

I'm passing a SR-IOV VF to an opnsense guest and other VFs from the same PF to ubuntu guests successfully. I'm using ubuntu 18.04 with XEN 4.9 as hypervisor. The network card is a 4 port Intel i350.

OPNSense and the ubuntu guests can communicate via the VFs. The only thing I couldn't get working is VLAN on top of the VF.

Kind regards
Torsten

[nzkiwi68]

Kind of an old topic, I know...

But I see SR-IOV is now supported on Hyper-V with FreeBSD
https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/supported-freebsd-virtual-machines-on-hyper-v


Anyone got this working?

[Ramalama]

Super old post, just seen 😂

However, im running opnsense as vm on Proxmox with an VF passed through.
The card is an X550-T2 which is onboard (asrock rack x570d4i-2t)

- A year ago, i had to use own compiled VF drivers from intel for Freebsd, because of some errors/performance/functionality i needed.

- Since opnsense 23?, im sorry i don't know exactly when it started to work flawlessly without my drivers or any modifications, but it's running absolutely flawless and everything works!
Vlans on top of the vf device works either without issues, even all hardware acceleration.
You just need spoochk=off and trust=on and in some cases set vlan=0 for the vf device in proxmox.
Basically vlan=0 deactivates vlan filtering, trust=on allows mac address changes and promiscuous and spoofchk=off is needed if you don't disable vlan filtering with vlan=0 and you use multiple vlans on the vf device not just one.

You can set only one vlan for an vf device, not multiple trunk vlans etc...
If you want spoofchk and hardware vlan filtering working, you need to passthrough multiple vf devices to opnsense, one for each vlan and assign all the vlans to vf devices accordingly with "ip link set enp35s0f0 vf 1 vlan 25" for example... And so on...

But anyway keep in mind, if you use the same primary function where you passthrough your vf and on the same primary function is a linux bridge "vmbrX" bound, you'll need to modify fdb tables.

For that reason i switched lately to passthrough the whole primary function to opnsense, because i have anyway 2 of them and if i use one 10gbe cable or 2 to the switch, doesn't matter for me.

The fdb table modification is easy tho, there are even automated scripts in proxmox forum for that...
It works good, but in my opinion it's easier for my specific situation to simply pass the pf.

However, the point is, VF works perfectly fine in opnsense nowadays.
So if you need it, do it!