OPNsense Forum

Archive => 22.1 Legacy Series => Topic started by: mater on February 01, 2022, 09:24:50 am

Title: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 01, 2022, 09:24:50 am
Hi,

I run an odroid H2+ with Proxmox and an opnsense VM.
I made an identical VM copy of my 21.7 install and upgraded it to 22.1, also installed os-realtek-re plugin.
So I can compare the performace between both installs, I can switch between them.

My Internet Connection is 1000/50 Mbits with a typical speed 940/52 Mbits.


The realtek nics are configured as follows:
re0 -> pcie passthrough (Proxmox) -> WAN opnsense
re1 -> pcie passthrough (Proxmox) -> LAN + VLANs opensense
usb nic -> management interface proxmox

Under the old install I can get the full speed, reliable!
With the new install I get unreliable speed between 50 - 830Mbits (strongly fluctuating).
Sometimes the speed drops abrubt to under 50Mbits.



So I tested another config, as follows:
re0 -> pcie passthrough (Proxmox) -> WAN opnsense
re1 -> bridge (Proxmox) VLans + Management interface proxmox -> LAN + VLANs opensense

With this config I have the same exact issue under 22.1

While testing I switched between both VMs, stopped one, started the other.
The issue is not the internet connection, it is rock stable!
The Issue with 22.1 is persistent.

Is there something, that need to configured different in 22.1 compared to 21.7.


And yes I asigned the vlans to parent interface, I hope  ;D

Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: elcocoloco on February 01, 2022, 11:13:06 am
Did you enable the interfaces also? Assigning is not enough I believe. So assign, enable in interfaces, save (you don't have to set anything)
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 01, 2022, 11:21:39 am
Yes the interfaces are enabled
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: elcocoloco on February 01, 2022, 12:29:21 pm
Yeah that should be enough, I'm out of ideas then, sorry
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 01, 2022, 05:21:14 pm
I tested the connection with iperf3.
The speed between opnsense 22.1 and my computer over the virtio nic is fine.

I think it could be 2 things:
the realtek driver
or
routing speed issue in 22.1
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: KHE on February 01, 2022, 05:58:31 pm
Hi,

according to the release notes, the realtek driver was changed. There is a way to get back the old one by installing the os-realtek-re plugin. I have no realtek NICs, so don't ask me more about it. ;)


KH
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 01, 2022, 06:42:02 pm
As written in my first post, I have installed this Plugin. Without the plugin the nic doesn't get recognized.
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: franco on February 01, 2022, 07:24:12 pm
The plugin is the same code entirely before and after the upgrade. I would still think there is an issue here that was documented in the release notes for 22.1. At least that would be my best guess.


Cheers,
Franco
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 01, 2022, 07:45:15 pm
I read the release notes, I don't know which issued you mean.
Can you please say which issue you mean?
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: franco on February 01, 2022, 08:50:14 pm
Could be https://github.com/opnsense/changelog/blob/70cd791f0528f26d2e804601f1eb6f55c384d3b7/community/22.1/22.1#L194

Almost a dozen recent reports have been solved this way.


Cheers,
Franco
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 02, 2022, 10:25:40 pm
Thank you for the link!

I have attached the pictures that all interfaces are assigned.
The parent interface "Management" is also assigned.

All Hardware Offload Settings are off.

Do I have missed something?
I didn't find any other offload settings.
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: franco on February 03, 2022, 01:51:18 pm
Hmm, did you do any MAC spoofing or use specific media settings for the Realtek links? And your last version was 21.7.8 which worked?


Cheers,
Franco
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on February 05, 2022, 11:23:34 am
I have a simlple setup. Only with vlans and some Firewall rules.
It is also running ddns and wireguard.

Yesterday the firewall crashed completely.


I have ordered a new device with Intel Nics to compare with the realtek nics.
But I think that gets delivered in 2 to 3 weeks to europe, because of chinese holidays.

I will further investigate and get updates to you.

Thanks.
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on March 20, 2022, 05:22:35 pm
Now, finally I got my new device with Intel NICs from china.
It has the exact same CPU and RAM config.

Now I cloned the System 1:1 to the new device and assigned the NICs.

I get full gigabit now!

@franco
So I think it is an issue with the new realtek driver.
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: franco on March 21, 2022, 07:58:39 am
> So I think it is an issue with the new realtek driver.

I guess you mean the FreeBSD driver. The thing is I think everybody knows that.


Cheers,
Franco
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: jclendineng on March 21, 2022, 02:47:47 pm
Now, finally I got my new device with Intel NICs from china.
It has the exact same CPU and RAM config.

Now I cloned the System 1:1 to the new device and assigned the NICs.

I get full gigabit now!

@franco
So I think it is an issue with the new realtek driver.

Be careful with those Chinese nics, usually they are samples or just mis-branded.  I have one of those and it works well, but the "intel" nics show up as Realtek in the interface overview but it works without the re drivers so they are clearly intel-ish.  Just saying, Chinese electronics are a black hole of false advertising, just be aware.  I have some interesting stories of weird CPU engineering samples I have from stuff like that ;D
Title: Re: Bad performance on Proxmox with Realtek NICs and/or virtIO Bridge
Post by: mater on March 22, 2022, 07:01:16 pm
Yes, but in OpnSense 21.7 I didn't have that issue with the same Realtek NICs.

So yes, Intel NICs are better in this regard  :)