Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge

[mater]

Hi,

I run an odroid H2+ with Proxmox and an opnsense VM.
I made an identical VM copy of my 21.7 install and upgraded it to 22.1, also installed os-realtek-re plugin.
So I can compare the performace between both installs, I can switch between them.

My Internet Connection is 1000/50 Mbits with a typical speed 940/52 Mbits.


The realtek nics are configured as follows:
re0 -> pcie passthrough (Proxmox) -> WAN opnsense
re1 -> pcie passthrough (Proxmox) -> LAN + VLANs opensense
usb nic -> management interface proxmox

Under the old install I can get the full speed, reliable!
With the new install I get unreliable speed between 50 - 830Mbits (strongly fluctuating).
Sometimes the speed drops abrubt to under 50Mbits.



So I tested another config, as follows:
re0 -> pcie passthrough (Proxmox) -> WAN opnsense
re1 -> bridge (Proxmox) VLans + Management interface proxmox -> LAN + VLANs opensense

With this config I have the same exact issue under 22.1

While testing I switched between both VMs, stopped one, started the other.
The issue is not the internet connection, it is rock stable!
The Issue with 22.1 is persistent.

Is there something, that need to configured different in 22.1 compared to 21.7.


And yes I asigned the vlans to parent interface, I hope 

[elcocoloco]

Did you enable the interfaces also? Assigning is not enough I believe. So assign, enable in interfaces, save (you don't have to set anything)

[mater]

Yes the interfaces are enabled

[elcocoloco]

Yeah that should be enough, I'm out of ideas then, sorry

[mater]

I tested the connection with iperf3.
The speed between opnsense 22.1 and my computer over the virtio nic is fine.

I think it could be 2 things:
the realtek driver
or
routing speed issue in 22.1

[KHE]

Hi,

according to the release notes, the realtek driver was changed. There is a way to get back the old one by installing the os-realtek-re plugin. I have no realtek NICs, so don't ask me more about it.


KH

[mater]

As written in my first post, I have installed this Plugin. Without the plugin the nic doesn't get recognized.

[franco]

The plugin is the same code entirely before and after the upgrade. I would still think there is an issue here that was documented in the release notes for 22.1. At least that would be my best guess.


Cheers,
Franco

[mater]

I read the release notes, I don't know which issued you mean.
Can you please say which issue you mean?

[franco]

Could be https://github.com/opnsense/changelog/blob/70cd791f0528f26d2e804601f1eb6f55c384d3b7/community/22.1/22.1#L194

Almost a dozen recent reports have been solved this way.


Cheers,
Franco

[mater]

Thank you for the link!

I have attached the pictures that all interfaces are assigned.
The parent interface "Management" is also assigned.

All Hardware Offload Settings are off.

Do I have missed something?
I didn't find any other offload settings.

[franco]

Hmm, did you do any MAC spoofing or use specific media settings for the Realtek links? And your last version was 21.7.8 which worked?


Cheers,
Franco

[mater]

I have a simlple setup. Only with vlans and some Firewall rules.
It is also running ddns and wireguard.

Yesterday the firewall crashed completely.


I have ordered a new device with Intel Nics to compare with the realtek nics.
But I think that gets delivered in 2 to 3 weeks to europe, because of chinese holidays.

I will further investigate and get updates to you.

Thanks.

[mater]

Now, finally I got my new device with Intel NICs from china.
It has the exact same CPU and RAM config.

Now I cloned the System 1:1 to the new device and assigned the NICs.

I get full gigabit now!

@franco
So I think it is an issue with the new realtek driver.

[franco]

> So I think it is an issue with the new realtek driver.

I guess you mean the FreeBSD driver. The thing is I think everybody knows that.


Cheers,
Franco