Setup Questions

[eminent]

Good evening,

I currently have TDS 1000 Mbps Down and 400 Mbps, no PPPOE.  Plugging the computer directly into the Fiber Box directly I get 900 to 970 Mbps consistently.  But when utilizing a router, really any router it slows to 300 to 350 Mbps. I have tried Asus RT-AX88U, Linksys Velop, and Netgear Orbi.  Which lead me to try Opnsense, after making the switch I love the features but I still cannot reach speeds past 400 Mbps down with speedtest straight from the CLI.

My current setup is a Qotom Intel i5-7200u, 6 Intel I211-AT, 8GB of RAM

I followed this guide here:
https://forum.opnsense.org/index.php?PHPSESSID=do37lppp3v4gidtdjsdl0do4bp&topic=6590.0

My current config settings for loader.conf.local is below.  But I still range 350 Mbps to 400 Mbps. Any advice would be appreciated, or just a point in the right direction.  Thank you.

Code Select Expand

#Agree with Intel license terms
legal.intel_ipw.license_ack=1
legal.intel_iwi.license_ack=1
#Check for interface specific settings and add accordingly.
# These ae tunables to improve network performance on Intel igb driver NICs

# Flow Control (FC) 0=Disabled 1=Rx Pause 2=Tx Pause 3=Full FC
# This tunable must be set according to your configuration. VERY IMPORTANT!
# Set FC to 0 (<x>) on all interfaces
hw.igb.0.fc=0 #Also put this in System Tunables hw.igb.<x>.fc: value=0
hw.igb.1.fc=0
hw.igb.2.fc=0
hw.igb.3.fc=0
hw.igb.4.fc=0
hw.igb.5.fc=0
hw.igb.6.fc=0
# Set number of queues to number of cores divided by number of ports. 0 lets FreeBSD decide
hw.igb.num_queues=0

# Increase packet descriptors (set as 1024,2048, or 4096) ONLY!
# Allows a larger number of packets to be processed.
# Use "netstat -ihw 1" in the shell and make sure the idrops are zero
# If the NIC has constant disconnects, lower this value
# if not zero then lower this value.
hw.igb.rxd="4096" # For i340/i350 use 2048
hw.igb.txd="4096" # For i340/i350 use 2048
net.link.ifqmaxlen="8192" # value here equal sum of above values. For i340/i350 use 4096

# Increase Network efficiency
hw.igb.enable_aim=1

# Increase interuppt rate
hw.igb.max_interrupt_rate="64000"

# Network memory buffers
# run "netstat -m" in the shell and if the 'mbufs denied' and 'mbufs delayed' are 0/0/0 then this is not needed
# if not zero then keep adding 400000 until mbufs are zero
kern.ipc.nmbclusters="1000000"

# Fast interrupt handling
# Normally set by default. Use these settings to insure it is on.
# Allows NIC to process packets as fast as they are received
hw.igb.enable_msix=1
hw.pci.enable_msix=1

# Unlimited packet processing
# Use this only if you are sure that the NICs have dedicated IRQs
# View the IRQ assignments by executing this in the shell "vmstat -i"
# A value of "-1" means unlimited packet processing
hw.igb.rx_process_limit="-1"
hw.igb.tx_process_limit="-1"

[bartjsmit]

That guide is for IPS. What speed are you getting with just the firewall pared down to a minimal configuration?

Bart...

[eminent]

Thank you for the response Bart,

First install, I followed that guide and had IPS enabled and was getting 330 Mbps to 400 Mbps.

I reinstalled to have a clean slate and try again.
First sets of speed tests ranged between 390 Mbps to 425. I then tuned to the following guide.

https://teklager.se/en/knowledge-base/opnsense-performance-optimization/

Which raised my speed tests ranged to 410 to 440.

[eminent]

Got TDS to come out to my location, they were able to get the TDS DSL Action Tech set up to use the fiber with near speeds as if the laptop.  The only difference was they put (null) for host, and they put tds for the domain.  All the other settings I have dug thru and cannot find a thing different.  The Null option does not work with OpnSense so i am going to test on other platforms.

Thank you everyone that read. And thank you Bart for responding.

[eminent]

Just wanted to update,

Discovered a guide to find MTU, which tested and messed round still wasn't working.  But ended up stumbling across this guide for peplink, which explains getting the ping method in addition to setting MSS. I am now getting near full speed on OpnSense.  I posted the Peplink link below if anyone is facing the same issue.

https://forum.peplink.com/t/how-to-determine-the-optimal-mtu-and-mss-size/7895

[trungveso]

https://forum.peplink.com/t/how-to-determine-the-optimal-mtu-and-mss-size/7895

Thank you for your sharing eminent!!!
I followed your link and throughput went up:

• from 3xx Mbits/s to 470 Mbits/s (setting MTU)
  
• from 470 Mbits/s to 600 Mbits/s  (setting MSS)

Hardware: t620 plus + i340-t4.