Intel i350 SR-IOV + VLAN does not work properly in OPNSense

[tokade]

Hi all,

I got the same problem with OPNsense 20.1.6 as mentioned in

...
Created SR-IOV virtual function, VF on physical interface enp5s0,
then tagged it with VLAN ID 20.
...
After passing Virtual Function device, it should have changed into untagged packet
But instead, it got VLAN ID 1024.

My setup is host Ubuntu 18.04 with Xen 4.9 and Intel i350 NIC

Created SR-IOV virtual function, VF on physical interface and tagged all VFs with VLAN ID 20. One VF is passed to OPNSense VM and a second VF is passed to another VM.

I tried with and without VLAN tagging in the VMs. Neither worked and there is "no" communication (PING, DNS,..) between the VMs.

Looking at the traffic in case no VLAN tagging in the VMs, which should be the right way using VLANs with SR-IOV. All traffic coming in to OPNSense VM is tagged with 1024.

Anybody got a similar setup with running SR-IOV and VLAN?

Kind regards
Torsten

EDIT:
tcpdump ubuntu VM

Code Select Expand

ping 192.168.20.1
PING 192.168.20.1 (192.168.20.1) 56(84) bytes of data.
18:06:03.256121 00:16:3e:a0:18:f1 > 00:16:3e:a0:16:f1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 18868, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.20.250 > 192.168.20.1: ICMP echo request, id 767, seq 1, length 64


tcpdump opnsense VM

Code Select Expand

18:10:09.974113 00:16:3e:a0:18:f1 > 00:16:3e:a0:16:f1, ethertype 802.1Q (0x8100), length 102: vlan 1024, p 0, DEI, ethertype IPv4, (tos 0x0, ttl 64, id 39437, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.20.250 > 192.168.20.1: ICMP echo request, id 778, seq 1, length 64


[tokade]

Hi all,

further investigation led me to a bug in freebsd "igb vf driver does not correctly handle vlan tag"
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209581

Is there any chance to fix that bug or a newer version of the driver?

Kind regards
Torsten