OPNsense Forum
English Forums => Hardware and Performance => Topic started by: foresthus on March 12, 2021, 12:46:10 pm
-
Hi,
I was wondering, why the perfomance using a realtec-nic is worse to an intel-nic. My bandwidth my provider supports is 600Mbit download and 150MBit upload. I used the configuration on two systems. The one with the intel-nic is 100% ok. The system with the realtek-nic less ... only 200MBit.
I am using a "RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller" on this system with the bad performance. Surly someone might think that both systems atre different in the haardware itself. That is correct. This is the benchmark of the used CPUs:
https://www.cpubenchmark.net/compare/Intel-i3-9100F-vs-AMD-Ryzen-5-3550H/3461vs3403
- Intel Core i3-9100F @ 3.60GHz = old System with intel-nic
- AMD Ryzen 5 3550H = new system with realtek-nic
I guess that is a problem with the driver, but I cannot see anything to tune this. I found a link via google search:
https://forums.serverbuilds.net/t/guide-resolve-realtek-nic-stability-issues-on-freebsd-pfsense-2-4-4-2-4-5-2-5-0-opnsense-use-2-5gb-realtek/3555
This not solves the problem. ;(
Anyone an idea? pfsense in there latest version 2.5 solved this.
Hope someone can help.
-
No one got an idea or a solution?
-
Hi,
Congrats to pfSense for "solving" this in 2.5.0.
Whoever wrote that guide isn't using OPNsense, because we replaced the standard FreeBSD driver in 2017.
https://github.com/opnsense/changelog/blob/1663700184747c800c64f5e009bcf857718fc292/community/17.1/17.1.2#L54
The guide and your assumption is factually wrong.
I'm not saying you have no issue, but the conclusion and fix are incorrect as this relates to stability not performance.
It's probably possible to tune the system to reach higher performance using sysctls but it's also true that likely the Realtek chip will be slower.
Cheers,
Franco
-
The Realtek NIC is more dependent on CPU performance. Have you tried disabling specter and meltdown mitigations? Also, make sure to enable powerd and set to “max” or “hiadaptive”. These two make the biggest impact in throughput performance.
https://www.reddit.com/r/OPNsenseFirewall/comments/mascfl/another_pfsense_refugee_slow_wan_throughput_where/
-
Hi,
I will try if your infos are helping. The se are my systems with opnsense with the same configuration:
I5 and opnsense and Intel-nic:
Probe URL: https://bsd-hardware.info/?probe=ef64f03609
AMD and opnsense and realtek-NIC:
Probe URL: https://bsd-hardware.info/?probe=4f3f6a4102
thnx for the replies.
-
Hi, and what is your solution?
Hi,
Congrats to pfSense for "solving" this in 2.5.0.
Whoever wrote that guide isn't using OPNsense, because we replaced the standard FreeBSD driver in 2017.
https://github.com/opnsense/changelog/blob/1663700184747c800c64f5e009bcf857718fc292/community/17.1/17.1.2#L54
The guide and your assumption is factually wrong.
I'm not saying you have no issue, but the conclusion and fix are incorrect as this relates to stability not performance.
It's probably possible to tune the system to reach higher performance using sysctls but it's also true that likely the Realtek chip will be slower.
Cheers,
Franco
-
What is the problem? The answer to your question is: yes, Realtek is not as good as Intel.
Cheers,
Franco
-
What is the problem? The answer to your question is: yes, Realtek is not as good as Intel.
Cheers,
Franco
Sorry that is not the answer for a solution. It is a not workaround or even an excuse.
-
It is actually. Buy Intel if it matters that much.
Cheers,
Franco
-
It is actually. Buy Intel if it matters that much.
Cheers,
Franco
This is a system with nic included:
https://store.minisforum.com/collections/amd-%C2%AE-ryzen-%C2%AE/products/deskmini-dmaf5-amd-ryzen-5-3550h?variant=35905326350497
Therefor no option to use an intel nic.
-
Sorry for being blunt, but you are being unreasonable.
If you can't replace the hardware or the NIC your best option is not to keep asking a community forum for help it can't give you. I am entirely unsure what you expect from it.
Cheers,
Franco
-
Do you know if MSI and MSI-X are enabled? On modern systems I've seen these being disabled result in odd transfer speed tests.
at the console query the following: sysctl hw.pci.enable_msi
sysctl hw.pci.enable_msix
Franco is correct in that realtek will never be as consistent as Intel. However, in the few times where I have to suffer through a realtek card, it can usually manage 600-700mbit with some regularity. Only 200mbits seems quite low for such modern processors.
When you run the transfer tests, how are you verifying the results? Are you pushing traffic through the OPNsense router from the WAN to the LAN side, or are you running an iperf instance hosted on one of the OPNsense interfaces?
-
I will test it and will be back.
-
@foresthus
These 1GbE Realtek NICs are troublemakers not designed for performance.
When you'll be testing, have a look on the CPU utilization of interrupt percentage.
You'll see a high number for your Realtek, very small for your Intel NICs. And this is exactly the problem -- CPU cores are very busy with the NIC instead of packets processing and other normal stuff (with or without MSI-X).
T.
-
I will test it and will be back.
The problem is you haven't actually posted what your issue is besides "this NIC is slower". It's a pretty well known fact that realtek NICs have inferior throughput, and are buggy in general. You say "Pfsense 2.5 fixed this" but don't actually link to a bug report or github source or performance testing or basically anything to backup that claim so it smells more like trolling than an actual request for help.
The link you provided is for loading newer versions of the realtek driver which people were using to address stability concerns, not throughput concerns. The driver in question is already included in opnsense:
https://github.com/opnsense/changelog/blob/da9944d43c1fe4466cab2e624727b1ad5f256ca9/community/20.7/20.7.4#L50
-
Hi Folks,
I now tested another configuration: https://bsd-hardware.info/?probe=57bd6d4d0c
The downloadrate is the same for INTEL-NICs with sensei included. My provider gives me a download up to 600MBit Download and 150MBit Upload.
New ideas?
root@opnsense:~ # speedtest-cli
Retrieving speedtest.net configuration...
Testing from SWN Stadtwerke Neumuenster GmbH (89.56.28.134)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Marco Bungalski GmbH (Verden) [133.13 km]: 8.853 ms
Testing download speed................................................................................
Download: 139.92 Mbit/s
Testing upload speed...
Upload: 131.67 Mbit/s
-
Can you try more in-depth testing to get an idea of where the bottlenecks are?
Run "top -aSCHIP" on the OPNsense router via an SSH session while also speed testing on clients behind the OPNsense router. Get traffic pushing through the router to simulate actual throughput instead of just a CLI speedtest on one interface of the router.
Monitor the top output and see if there is a process(s) that is loading up and causing a bottle neck.
Based on the output provided from the CLI test, it looks like it's just running a single thread. That could very likely be your bottleneck right there.
Try to check multithreaded speeds. https://www.dslreports.com/speedtest is a good test to check. I would also recommend downloading Linux ISOs via torrent, that's a good way to test if you can max out your connection speed.
-
Hi, after replacing my intel-nics to a newer version, the problem with the download and upload-rate is not that good as I wished to have.
NOW:
https://bsd-hardware.info/?probe=d8b645d95d
>> 82576 Gigabit Network Connection: The device is supported by FreeBSD versions 7.1 and newer.
BEFORE:
https://bsd-hardware.info/?probe=57bd6d4d0c
The device is supported by FreeBSD versions 6.1 and newer.
I hope they will fix it.
Therefor this is aother proof, that the addon sensei/sunvalley has got the problem with using net_map.
-
https://forum.opnsense.org/index.php?topic=22019.msg110532#msg110532
-
Hmm, you didn't mention before that you were using Sensei? That's a major factor that would have been good to know. ;)
You need to re-baseline and just run the performance tests with Sensei disabled and no other IPS/IDS packages running. If you have a traffic shaper configured, turn that off too.
Then re-run a download test with your newest (igb) network adapter and report back the results. I would suggest either using two iperf clients to push traffic through the firewall, or use a single client on the LAN side to download several linux ISOs via torrent. This should max out your connection and give you a good idea of what the max throughput will be.
While doing the above tests, also watch the output of this command at the SSH console: top -aSCHIP
Screenshot the CPU usage of that console when the throughput tests are running.
-
Hmm, you didn't mention before that you were using Sensei? That's a major factor that would have been good to know. ;)
You need to re-baseline and just run the performance tests with Sensei disabled and no other IPS/IDS packages running. If you have a traffic shaper configured, turn that off too.
Then re-run a download test with your newest (igb) network adapter and report back the results. I would suggest either using two iperf clients to push traffic through the firewall, or use a single client on the LAN side to download several linux ISOs via torrent. This should max out your connection and give you a good idea of what the max throughput will be.
While doing the above tests, also watch the output of this command at the SSH console: top -aSCHIP
Screenshot the CPU usage of that console when the throughput tests are running.
I will test that and will be back. thnx 4 the ideas.
-
@foresthus how did you go? I looked up the specifications of your machine and it looked ok, so I'm surprised you had issues.
Any progress?
-
I've been running OPNSense on a Minisforum GK41 for about two weeks now. It contains 2 RTL8111/8168/8411 PCI Express Gigabit Ethernet Controllers and a Celeron J4125.
I haven't had any issues maxing out a 1Gbps symmetrical PPPoE pipe. It's pretty close to maxing out a single core at 1Gbps, as expected. Enabling the NIC hardware features (CRC, TSO, LRO) made a minimal impact but I've had no issues with stability or performance.
If you're on PPPoE you will want to choose a CPU with good single core performance, even paired with an Intel NIC.
-
I've been running OPNSense on a Minisforum GK41 for about two weeks now. It contains 2 RTL8111/8168/8411 PCI Express Gigabit Ethernet Controllers and a Celeron J4125.
I haven't had any issues maxing out a 1Gbps symmetrical PPPoE pipe. It's pretty close to maxing out a single core at 1Gbps, as expected. Enabling the NIC hardware features (CRC, TSO, LRO) made a minimal impact but I've had no issues with stability or performance.
If you're on PPPoE you will want to choose a CPU with good single core performance, even paired with an Intel NIC.
Thank you for your response, I think the processor I'll end up using is this one
https://ark.intel.com/content/www/us/en/ark/products/97121/intel-core-i5-7500t-processor-6m-cache-up-to-3-30-ghz.html
With 4 or 8GB
I'll be using about 40 clients on it (but only 4 or 5 heavy hitters, like PS5, Desktop PC)
I am curious about this Sensei plugin so I'd like to use that.
Looking like an M.2 E Key, Realtek RTL 8111F I think ($30) plus the Intel i219LM on the mainboard.
Would this be reliable and ok?
-
The 7500T should be quite fast for firewall duties. If you can, I would really try to avoid any realtek NICs. Obviously if you have already purchased the hardware, you are stuck with what it can do. But given that you're in the process of spec'ing out a new solution, just avoid Realtek from the beginning and it'll be very trouble free.
A lot of folks still use realtek NICs and don't have issues. However if you get to choose from the start, I think we'd all admit they wouldn't be our first choice.
-
The 7500T should be quite fast for firewall duties. If you can, I would really try to avoid any realtek NICs. Obviously if you have already purchased the hardware, you are stuck with what it can do. But given that you're in the process of spec'ing out a new solution, just avoid Realtek from the beginning and it'll be very trouble free.
A lot of folks still use realtek NICs and don't have issues. However if you get to choose from the start, I think we'd all admit they wouldn't be our first choice.
I actually don't have the E Key eithernet adapter. It looks like it's in excess of 80$ US to get a single, Intel E Key M.2 2230 adapter which would fit in a micro PC like that, with shipping.
It's about $25 US to go Realtek and some people report no issues at all. I'm leaning towards taking the risk on Realtek 8111F