25GB Opnsense Newbie

[cryoplay]

Hi there.

Im an absolute Newbie in Linux / BSD / Opnsense, so please dont bite me.

Ive got a 25GB/s symetrical Fiber connection from my ISP and i tried to make an Opnsense Router because i doenst find a payable Router that can handle at least 15GB/s....

I have build a Test System:

Intel 9900
32GB Ram
Samsung nvme 1tb
Intel xxv710-DA2 NIC

Setting up opnsense and my Speeds are about 7GBs Down and 3GBs up.
I tried different tuneables but... im not sure if i make it right.

Can someone help and explain me how to find the Problem and solve it?

[lilsense]

Are you testing your speed from the test system? is that the OPNsense router?

[cryoplay]

Im testing it from a computer in the lan to speedtest.net.
Speedtest Server is the one from the ISP.

Managed to install Speedtest on the Opnsense Box.
CLI SSH testing:
Speedtest by Ookla

      Server: Init7 AG - Winterthur (id: 43030)
         ISP: Init7 (Switzerland) Ltd.
Idle Latency:     0.87 ms   (jitter: 0.03ms, low: 0.84ms, high: 0.93ms)
    Download:  8876.95 Mbps (data used: 5.3 GB)                                 
                  3.56 ms   (jitter: 1.50ms, low: 0.59ms, high: 18.97ms)
      Upload: 17889.00 Mbps (data used: 23.7 GB)                               
                  0.73 ms   (jitter: 0.37ms, low: 0.56ms, high: 6.20ms)
Packet Loss:     0.0%

[lilsense]

Are you able to see speed test at around 25Gbps speed without the OPNsense? If so, have you turned off Zenarmour/Sensei/IDS/IPS for the testing purpose?

[cryoplay]

Thank you for your reply.

I only see 25 Gb/s when i direct connect my Computer to the Fiber connection.
Without the opnsense box between.

Im not sure how i can turn off Zenarmour ( deinstal it? ).

I also found a Thread with a similar Problem.
https://forum.opnsense.org/index.php?topic=24302.0

My nic isnt at the latest firmware but i dont managed to update it.

Regards

[lilsense]

Glad you found the answer. Just research on how to them or ask the person who has done them to assist you with it.

[cookiemonster]

Im not sure how i can turn off Zenarmour ( deinstal it? ).

There is a page in the plugin UI where you can stop and bypass for testing purposes. Bypass is enough for a quick test.

[cryoplay]

Hi there.

No difference if Zenarmour is bypassed, in passive mode or on.

[cryoplay]

Glad you found the answer. Just research on how to them or ask the person who has done them to assist you with it.

I tried to contact the User about a Week ago. No Answer.

[cookiemonster]

If you're looking for the latest firmware for the card, look in ark. Be mindful, it's a low level flash operation. Don't do it unless you feel confident. https://ark.intel.com/content/www/us/en/ark/products/95260/intel-ethernet-network-adapter-xxv710da2.html
Best to try an elimination process.
Definitively remove any inline processing first like Zenarmor and Suricata.

[cryoplay]

Thanks for your replys.

Finding the Packages is not a Problem, thank you.
Downloading direct to the opnsense Box and starting the update is the Problem.

Sure, i know i can dissassemble the box and do the update on a Windows Machine. But i want to learn, so i have to figure it out how i do it direct on the box.

Same with the Speeds. I see theres some appliances to buy that can handle 25Gb/s but i think i have more CPU Power that the appliance so it should work.

I tried it on another Workstation with the Same NÎC but with Dual Xeon 2680V4 and 64GB Ram. Clean install and direct Testing without any changes and i got about 2Gb/s..

Can someone help me with the commands for downloading and installing the Intel NIC Firmware and drivers?

[cookiemonster]

I could give you a hand if this is your first time updating the firmware of a card as long as you accept all responsibility if things go wrong. A bad flash can render the card unoperable and permanently. It is easy but risky. Think if it as essentially upgrading the bios on a pc's motherboard. It's exactly the same.
Assumptions: a linux pc being used and you have a user called cryoplay created in OPN that is in the administrators group i.e. can use sudo). I also assume your OPN has the ssh service running and the ip is 192.168.10.1
The steps are:

1. finding the right downloads: the utility and the manual. The link above has them both. Download to your pc. Put it in say /home/cryoplay/Downloads/
You seem to going to be getting a compressed file "700Series_NVMUpdatePackage_v9_01_FreeBSD.tar.gz". Therefore your path is /home/cryoplay/Downloads/700Series_NVMUpdatePackage_v9_01_FreeBSD.tar.gz
2. Then transfer from pc to router machine. Use scp.
Drop to a shell in your pc, change directory to /home/cryoplay/Downloads/ by issuing $

Code Select Expand

cd /home/cryoplay/Downloads
Now you are in the Downloads directory, you can scp the file to opn:
$

Code Select Expand

scp 700Series_NVMUpdatePackage_v9_01_FreeBSD.tar.gz cryoplay@1921.68.10.1://tmp You'll need to provide your password set in OPN.
3. Now that the file is uploaded, you need to connect to OPN to continue there.
ssh to it and go to the /tmp directory where you uploaded the file.
4. Make a directory to work from: mkdir -p /tmp/flashwork
5. Uncompress the uploaded compressed file into the new directory: $

Code Select Expand

tar -xvf 700Series_NVMUpdatePackage_v9_01_FreeBSD.tar.gz -C /tmp/flashwork
6. Change directory to work from it where all the files now are $

Code Select Expand

cd ./flashwork
7. Now you follow the steps in the document.

[cryoplay]

Thank you very much for the Help cookiemonster.

Thank you very much for explaining this to me so well and precisely.

You helped me really much. All worked.
Now i learned how to transfer Files betwen my PC and the opnsense Box.
I doesnt found a way. I tried it days before and read so much, but all tutorials
that i found startet when the Files are allready on the Box.

Im now on point 7 and try to install the Firmware.

I have to figure it out how to start nvmeupdate64e.
I always get the error command not found because i doesnt know i need ./ to start nveupdate.

Now all worked and the Firmwareupdate is installed.
But how can i check and / or updating the intel NIC Drivers?

[cookiemonster]

According to the manual for the nvme update tool, when it is launched, it will list the adapters it finds in the system and prompts to update if there are updates available. That means evidently that the system needs to be able to connect to the internet.
Don't skip steps. Section 4 shows how to identify the device (the nic) and the current firmware version.

[cryoplay]

Everything worked.

Firmware update and drivers are installed.

What I did is the Intel E810 card installed instead of the E710 and I also installed firmware and drivers there.

Performance is worlds better.
9Gbps ​​down and up.

https://www.speedtest.net/my-result/d/9b8a6daa-0dfb-47f7-95f1-3e9f71f953d3

What I don't understand in the other thread is that with the DDP packages.

What I also have to say, in the speed test I have a maximum CPU load of 15%. Shouldn't there be more?

I don't understand how an opnsense appliance with, in my opinion, less power should be able to do over 20Gbps.