[Solved] VLAN Interface Errors

[ajm]

(Marked as Solved as its not an OPNsense issue but a FreeBSD one and not service-affecting anyway)

On a new install of 22.1, with six igb interfaces, five of which have 2 or 3 VLANs configured, I am seeing non-zero 'output errors'. These are on the VLAN interface not the physical interface.

Pic of Interface Statistics

The error counters are incrementing with traffic.
The counters are remaining at zero, on interfaces WITH a cable connected.
The counters are non-zero, on interfaces WITHOUT cable connected (igb2, igb3).
The counters are remaining at/near zero, on interfaces attached to VLAN's which are currently 'inactive' (no broadcast traffic).

They appear similar to those reported in:

https://github.com/opnsense/src/issues/74

However I'm NOT certain the errors I'm seeing are the same cause. I'm happy to assist with any further system info need to help find a fix. I'll open a new Issue if needed.

[ajm]

(I corrected a few typos, so if replying, please refresh your page to make sure you have the final version of the post. Apologies..)

[5SpeedFun]

Is this a physical machine or vm ?

In either case, do you see the same thing if you go to interfaces -> settings -> disable vlan hardware filtering?

[ajm]

It's a real machine. I checked the GUI and its set at the default, of 'Disable Hardware Filtering'.

However in the shell, ifconfig reports that the following options REMAIN ACTIVE on all the interfaces configured with VLAN's, and have only been disabled on the interfaces which is being used untagged:

igb0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500              options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 00:0d:48:47:f9:b1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN
        options=4800028<VLAN_MTU,JUMBO_MTU,NOMAP>
        ether 00:0d:48:47:f9:b2
        inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

igb0 is the parent of three VLAN interfaces. igb0 itself is not configured as an Interface.

So it seems that the GUI setting is not being correctly applied to all interfaces. Hmm..

[Cuffs]

igb0 itself is not configured as an Interface.

This might be the problem

Assign igb0 without ipv4 or ipv6 settings.

I guess then the GUI settings will work

cheers
Christian

[franco]

Yes, since 22.1 it's required to assign and enable the parent interface without further configuration in order to handle hardware checksum settings.

Unfortunately they seem to be mostly broken still in its enabled state and don't do anything for forwarding traffic. But at least it's easy to fix by configuration.


Cheers,
Franco

[ajm]

OK, thanks.

In order to 'Disable VLAN hardware filtering' correctly on the VLAN parent interfaces (which I cannot use due to some other bug in BRIDGE(4) ), I added them all as 'Interfaces', unconfigured but enabled.

I then verified using ifconfig that these interfaces did indeed have the HW VLAN options disabled.

However, I am still seeing the spurious 'output errors' on the VLAN interfaces. I also tried 'Enable VLAN hardware filtering', but that made no difference either.

I will boot the machine up into vanilla FreeBSD 13.0 and see how looks there.

[ajm]

I booted the same hardware with 13.0-RELEASE-p7, with the same effective interface/vlan/bridge configuration as being used in the OPNsense instance.

I also had to set in /etc/rc.conf the following options on all six igb interfaces, to workaround the issue described in this bug, which I understand is fixed in STABLE but not RELEASE:

"-rxcsum -txcsum -rxcsum6 -txcsum6 -tso6 -tso4 -lro -vlanhwtag -vlanhwfilter -vlanhwcsum -vlanhwtso up"

Using ifinfo, I am seeing similar 'output errors' on the VLAN sub-interfaces as under OPNsense, as described in my first post in this thread.

So it appears that whatever issue is the cause, is due to FreeBSD, and is present not only in 13.0-RELEASE, but 13.0-STABLE.

I think I will just 'move-on' now to other things, as these errors are an annoyance only, as far as I can tell they're not impacting on function.

[ajm]

I did a tcpdump on one of the bridge member ports which is incrementing 'output errors', per ifinfo.

As already mentioned, the member port is a VLAN interface, and the parent interface is in a no carrier state.

The traffic hitting the member port outbound, is IPv4 & IPv6 broadcast & multicast traffic.

For some reason, the bridge is forwarding broadcast & multicast to all member ports, regardless of the link state of the parent interface !

Any ideas on how to stop it doing this will be gratefully received..