Checksum issues with VirtIO in QEMU/KVM environment and OPNsense 22.1

[bob4os]

I'm running OPNsense 22.1 in a QEMU/KVM environment - pc-i440fx-3.0 architecture and have some major networking issues since updating yesterday (and once again I forgot to take a snapshot before updating).
(Ryzen 2600, 128 Gb RAM, B450 Chipset, Mellanox ConnectX-2)

All interfaces are VLAN on vtnet0.

Disabling rxcsum, rxcsum6, txcsum and txcsum6 got at least communication between VLAN up again.

Code: [Select]

ifconfig vtnet0 -rxcsum -rxcsum6 -txcsum -txcsum6Until rebooting - is there a permanent solution to this?

Before only ICMP (ping) and UDP passed through my firewall rules.
TCP packages failed due to checksum errors (don't remember which log), connections were visible in the firewall "Live View", but nothing went through.

I think, this is related to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165059 .
Have there been any changes in handling VirtIO adapters?

I have no additional "Tunables" set other than default (I reset them), "Disable hardware checksum offload",  "Disable hardware TCP segmentation offload" and "Disable hardware large receive offload" are checked, "Enable VLAN Hardware Filtering" enabled.

Right now the OpenVPN connection is pretty much unusable and I'm not on location, general performance is abysmal.

Any advice?

[bolmsted]

Seems like similar symptoms as here
https://forum.opnsense.org/index.php?topic=26602.0

I'm also running in Proxmox using Virtio so I will be paying attention to this thread if it gets traction.

[franco]

Please assign vtnet0, enable and leave the rest as is. Done.


Cheers,
Franco

[yolocoffee]

I had similar issues and recreating the opnsense vm using Q35/OVMF fixed the issue for me. Granted I'm passing through two interface cards for LAN and WAN and just using the bridged vtnet0 for accessing the opnsense GUI from the host in case shit hits the fan.

[bob4os]

Please assign vtnet0, enable and leave the rest as is. Done.


Cheers,
Franco

Thanks for replying - and helping

Ok, looking good now - I forgot to enable the interface.
(Those who can read are at a clear advantage.)

The interface options rxcsum and txcsum are disabled by default now (after rebooting).

I guess that's what the release notes meant with interface media settings...

[franco]

I agree the wording is unclear and we will likely improve it. Thanks for reporting back


Cheers,
Franco

[stapel]

Franco, thanks for clarifying this. bob4os I appreciate the detail you provided, it allowed me to identify the exact same issue in my case.  Hopefully people find this article.

If you're reading this thread and a little confused like I was initially, here's a video going in depth on this issue and the easy fix described by Franco above:

https://youtu.be/69cNH9UX_es

[tokade]

I'm using opnsense as VM under XEN (ubuntu 20.04). I did assign and enable the parent interfaces for the xn interfaces as described which only used for VLAN.

How about the parent interfaces for pci passed through nics which are only used with VLANs? Shall those interfaces be assigned and enabled too?

Regards
Torsten

[franco]

It's recommended to assign and enable these as well. If it works without assignment that is good, but future OS changes might change this as drivers get more checksum offloading capabilities or something is actually broken with them.


Cheers,
Franco

[tokade]

Thx, will do that.

[badsmoke]

hello I am also on proxmox on the road with my opnsense, after the update to 22.1 is the upload totally collapsed to about 2-5mbits, as in a few other threads also already seen.


I have all interfaces (always) assigned but the problem still exists.

virtuio, q35

[TheHellSite]

This change is permanent. Previously if you had more than one VLAN and modified settings from it that affected the parent all the VLANS tried to apply their settings to the parent which is undefined behaviour solely dependent on the order of the interfaces in the configuration.

For MAC addresses the situation was even worse...


Cheers,
Franco

Hello Franco,

firstly thank you for your hard work in making OPNsense greater with every update.


Now back to topic...

My Setup

I am also running 2x OPNsense virtualized on two different Proxmox hosts both VMs with only VirtIO NICs.
One Proxmox system has an AMD CPU and the other an Intel CPU.

Both OPNsense VMs have the CPU type set to "host".
"If you want an exact match, you can set the CPU type to host in which case the VM will have exactly the same CPU flags as your host system." - PVE Wiki

vmbrX is the interface bridge on the Proxmox host.

- vmbr2=vtnet2 is the "LAN" interface and has many VLANs and the parent interface (vtnet2) is also assigned to a subnet.
--> of course no issues should occure

- vmbr1=vtnet1 is being used to access the Modem webinterface, directly assigned.
--> of course no issues should occure

- vmbr0=vtnet0 is my "WAN" interface.
This has been assigned vtnet0_vlan7 (Telekom VDSL) directly on the OPNsense and not on the Proxmox host interface settings. It then has been set to PPPoE mode to establish the WAN connection.
In case this matters: My WAN speed is 100/40 Mbit.


My question

What I don't understand is why am I not experiencing any issues?
Is my WAN speed to slow to notice the issue?
I am doing pretty much the same as all the others.

The only interface left unassigned is my vtnet0 WAN interface. So I should in theory experience the issues others are having.

Find attached my interface config.

[badsmoke]

I have found my problem, the vlan interface was indeed assignment but not enabled, now goes with me with the new version also fast again