Intel i350 SR-IOV + VLAN does not work properly in OPNSense

[park0kyung0won]

Hello
My setup is:

Host machine: Arch Linux + KVM, has Intel i350 NIC.

Created SR-IOV virtual function, VF on physical interface enp5s0,
then tagged it with VLAN ID 20.
```
echo 1 > /sys/class/net/enp5s0f0/device/sriov_numvfs
ip link set enp5s0f0 vf 0 mac blahblah
ip link set enp5s0f0 vf 0 vlan 20
```
Passed this VF to OPNSense VM.

Also I created VLAN interface on the same port, which is enp5s0,
tagged with VLAN ID 20.
```
ip link add link enp5s0f0 name VIRT type vlan id 20
```
Created bridge on top of this VLAN interface VIRT, for other VMs(Let's call them clients).

I could capture packets on three places: Client VM, OPNSense VM and Host physical interface(enp5s0f0).


When OPNSense VM sends packet to the guest VM(ping):

On OPNSense VM: ping packet is untagged
On Host enp5s0f0: ping packet has VLAN ID 20 (Which is expected, for VF is tagged with VLAN ID 20)
On client VM: ping packet is untagged (Which is expected, for interface that libvirt network bridge is sitting on(VIRT) is VLAN interface on enp5s0 with VLAN ID 20)

Here is the problem.

When client VM sends DHCP request to OPNSense VM,
On client VM: DHCP Req packet is untagged
On Host enp5s0f0: DHCP Req packet is tagged with VLAN ID 20 (Expected)
On OPNSense VM: DHCP Req packet is tagged with VLAN ID 1024 (??)

DHCP Req packet had VLAN ID 20 at the moment it was passing Virtual Function for it passed VF with VLAN filter of VID 20.

After passing Virtual Function device, it should have changed into untagged packet
But instead, it got VLAN ID 1024.
There must be something wrong with igbvf driver with OPNSense I suspect

[mimugmail]

Can you try with 19.1.10 too? There were some reports about problems with combinations of I350, QEMU and 19.7