None of my VLAN interfaces are working after upgrade to 22.7

[IsaacFL]

Just what the subject says.  No connectivity at all to opnsense from VLAN interfaces.

WAN  ix0    works
LAN   em0  works

VLANs all on ix1, none work, but they show on list of interfaces as up.

[IsaacFL]

Trouble shooting:
reboots, didn't help.

What fixed it. 
Interfaces: Settings
Changed VLAN Hardware Filtering from Disabled to Default
Changed Disable hardware checksum offload from unchecked to checked.

Reboot, now is working. Not sure what fixed it though.

[annoniempjuh]

i had the same problem, after changing VLAN Hardware Filtering from Disabled to Default and reboot its working again.
hardware checksum offload was already disabled.

i use PPPoE with vlan on a intel nic with IX driver

[franco]

These are quite old installations or at least older configuration files from previous installations?

At some point we changed the default from "disabled" to "default" because it appeared more stable. Must have been another change in FreeBSD 13.1 pulling that string.


Cheers,
Franco

[IsaacFL]

These are quite old installations or at least older configuration files from previous installations?

At some point we changed the default from "disabled" to "default" because it appeared more stable. Must have been another change in FreeBSD 13.1 pulling that string.


Cheers,
Franco

I think I may set it to disabled at some point. I just got the 10G NIC a few months ago and wasn't sure what it should be set at or what the difference was.

[a3w]

Trouble shooting:
reboots, didn't help.

What fixed it. 
Interfaces: Settings
Changed VLAN Hardware Filtering from Disabled to Default
Changed Disable hardware checksum offload from unchecked to checked.

Reboot, now is working. Not sure what fixed it though.

Thanks! I upgraded and had the same issue. Made the change from disabled to default within interface>>settings>>from disabled to default. Next, I rebooted and everything worked.

UPDATE
It appears that the VLANS work shortly after a reboot but within a min or two its broke - this latest update has broken all VLANs. Next, troubleshooting option is to install a clean OPNsense instance :(

UPDATE (1 August 2022)
A clean installed fixed/resolved my issue with VLANS not working...not sure why but oh well.

[franco]

Just to add those data points here is the commit I was referring to:

https://github.com/opnsense/core/commit/e2f6272957d8f3e

Looks like 20.7 was the first OPNsense version that shipped the good "default" value setting. Older installs may have the suboptimal "disabled" or if manually adjusted like stated above.

Not an awful lot we can do if such NIC settings disrupt basic connectivity. I think in some cases the VLAN filter capability is required for VLAN to work at all which seems counter-intuitive or at least the driver fails to set that capability when a VLAN is configured on it. I think this also happened to another driver in 22.1 and got subsequently resolved in a 22.1.x via FreeBSD fix.


Cheers,
Franco

[tessus]

@franco does this mean people should always use "Leave default" for VLAN Hardware Filtering or does this depend on the NIC?

Additionally all these HW offload settings are a bit vague. IMO all newer network cards should support these, but it seems to be a gamble to actually enable them. In theory it should be a lot better to let the HW handle stuff, but according to many posts it's the opposite.

[ticker]

Thank you, changing from Disabled to Default fixed my issue as well!

[franco]

@tessus yes it depends on the NIC and apparently if drivers are updated it can change for existing NIC so it changes with FreeBSD updates (unfortunately).

[tessus]

Thanks @franco. In that case I will change my setting to "Leave default".

Do you have any suggestions for the other options (Hardware CRC, Hardware TSO,
Hardware LRO)? Or do I have to test them individually? (I'm using I211 1GB - igb0)

P.S.: I have just noticed that my questions could be considered as hijacking this thread. If so, please let me know and I open a new topic.

[franco]

@tessus since issue is solved follow up question are useful I think :)

It really really depends on the driver at hand, maybe even the hardware the driver is operating. I can only encourage you to try using the options to see if your situation improves or worsens. New OS updates can always add new stuff or break existing acceleration for particular hardware.

Since 22.1 we also have interface-specific hardware settings available if you are in a mixed NIC environment and should be preferred over the global settings (which are mostly for avoiding issues on a global scale).

Can't give more specific advice on this than that. Hope it helps anyway.


Cheers,
Franco

[XeroX]

Hello guys.

I still have issues with VLANs.

None of my VLANs are working after the upgrade.

I switched to "Leave Default" prior updating (as I've an older installation).

Suricata runs on LAN (igb1), WAN(igb0) (Physical interfaces). VLANs are on igb1.

All Hardware acceleration is off (switched off CRC Offloading recently).

My VMs are still not able to reach the gateway (no route to host).

Any advices?

EDIT: 100% IPS related. However I'm not sure why this was working before with 22.1 (VLAN set to leave default) and isnt working with 22.7. Downgrade to 6.0.5 did not work as well.

[XeroX]

Okay so "Leave Default" can be Enabled or Disabled. Depends on the driver.

However "Leave Default" is Enabled for igb (i211-AT). You have to set it to Disabled to make VLANs working again. Really confusing and really shitty implementation on BSD side.

This effect only applies with IPS enabled. Enabled, Disabled or Default work without Suricata.

[tessus]

What is i211-AT? I only see i211 on my system. So maybe I should not set it to "Leave default" after all.

I am currently travelling and won't be back for 2 months, so I will wait until I am home before updating to 22.7 and/or playing around with those settings...