ixv virtualized NIC (KVM with SR-IOV) broken in 22.7.5

[satwell]

I run OPNsense in a Linux KVM virtual machine with an Intel X520-D2 NIC.  I use SR-IOV with the NIC and provide a single VF to the OPNsense VM.  I use multiple VLANs on the VF.

This was working fine in 22.7.4.  But after upgrading to 22.7.5, I can't send or receive packets on any VLANs.  And it's still broken with 22.7.7.  I've rebooted the host, and I've rolled back/re-upgraded multiple times.  I'm pretty confident that it's a change in 22.7.5 that broke it.

No obvious error messages on the OPNsense console.  But on the Linux host, I'm now getting these kernel messages when booting OPNsense:

Code Select Expand


[  +1.841648] vfio-pci 0000:02:10.2: enabling device (0000 -> 0002)
[ +14.454283] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.011681] ixgbe 0000:02:00.0 enp2s0f0: Unhandled Msg 00000010
[ +13.688786] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.071553] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.016387] ixgbe 0000:02:00.0 enp2s0f0: Unhandled Msg 00000010
[  +0.089071] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.071373] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.012678] ixgbe 0000:02:00.0 enp2s0f0: Unhandled Msg 00000010
[  +0.104481] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.073417] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.015499] ixgbe 0000:02:00.0 enp2s0f0: Unhandled Msg 00000010
[  +0.105531] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.071245] ixgbe 0000:02:00.0 enp2s0f0: VF Reset msg received from vf 1
[  +0.015634] ixgbe 0000:02:00.0 enp2s0f0: Unhandled Msg 00000010
[  +1.808707] ixgbe 0000:02:00.0 enp2s0f0: 4 Spoofed packets detected
[  +2.015959] ixgbe 0000:02:00.0 enp2s0f0: 1 Spoofed packets detected
[  +2.015956] ixgbe 0000:02:00.0 enp2s0f0: 1 Spoofed packets detected


The spoofed packets message repeats.  I'm guessing from DHCP attempts on the WAN VLAN.

Any idea what could be wrong, or ideas to try?

[satwell]

I've been able to narrow down the problem to VLAN tagging.  If I reconfigure the hypervisor to tag the virtual interface for a specific VLAN and disable VLAN tagging in OPNsense, I get a working network interface.  But obviously for that specific VLAN only, so this isn't a work-around.