Built on N150

[BrandyWine]

Built my OPNsense on this N150 device (US amazon item B0F4WXKZRB bare bones)
16GB ram, 256GB NVMe drive
A little tunables tuning and it does excellent job for home 1.3Gb copper service. Maybe some day I get docsis4, then I switch the fw WAN over to a 10G gbic.
Temps stay between 46-50C.
Odd though, the bottom cover has tapped holes (possibly for mounting, not sure), but I plan to install another micro fan there.
So far so good.

Here's a set of tuning which yields decent performance with a balance between idle state and max processing (heat). Most of these tunables were not in the system, but still try and find each before adding it.

# this removes some time consuming resources from entropy for random
kern.random.harvest.mask: 65887

# lower means leans towards higher performance. setting 0 seems to peg cpu at max freq
# if you want to unbalance them for power savings, set 0 around 10 and the others at 50 or 75, etc
dev.hwpstate_intel.0.epp = 25
dev.hwpstate_intel.1.epp = 25
dev.hwpstate_intel.2.epp = 25
dev.hwpstate_intel.3.epp = 25

# leave 0 alone at C1 (default C1, so no need to set it)
dev.cpu.1.cx_lowest = C2
dev.cpu.2.cx_lowest = C2
dev.cpu.3.cx_lowest = C2

# place this KLM load in about middle of the '/usr/local/etc/rc' script
kldload cc_htcp

net.link.ifqmaxlen=2048
net.inet.tcp.cc.algorithm=htcp
net.inet.tcp.cc.htcp.adaptive_backoff=1
net.inet.tcp.cc.htcp.rtt_scaling=1
net.inet.tcp.rfc6675_pipe=1
net.inet.tcp.syncache.rexmtlimit=0
net.inet.tcp.syncookies=0
net.inet.tcp.isn_reseed_interval=4500
kern.random.fortuna.minpoolsize=2048
machdep.hyperthreading_allowed=0

[tn881023]

Thanks Brandywine, I'm interested to try these as I'm running an N150 box myself but could you please provide a little more context on what they do?

Thank you!

[BrandyWine]

I do recommend you look up each of them, just to get a general overview.
Remember, the OPNsense fw itself is a router for the most part, so performance tuning a software "router" is key to fast routing. Tuning for TCP/IP is not a big worry because there's little to no connections landing on the fw IP stack itself (no sockets being created, etc). I got most of those tunables by looking up "performance tuning freeBSD router". And recall, everything that goes through a TCP/IP based router, has a lot of settings in the TCP/IP realm, but some won't be relevant.

Then you have the hardware, you can manipulate a few things there. Many play around with powerd to try and get least power usages when the system workload is low. I don't use any BIOS or powerd for power tuning.

dev.hwpstate tells system where in it's (cpu) range of Hz it should hang around by. Set it to zero and the cpu will run at max freq all the time (kinda not useful, wasting power, making heat, etc). 50 means hang around the middle. I think on my N150 set to 25 the cpu's hang around 1.2GHz, set at 50 the cpu's stayed around 800MHz, they can boost to 3.6GHz as needed. My testing did not reveal a meaningful linear scale. 25 seems to be ok.

CPU CX is like a power profile. C1 C2 C3. C1 keeps things very active, C3 can allow the cpu to basically sleep, C2 is in the middle. I keep cpu0 active so that it's ready to take on new instructions, and the others C2 so they can relax some (partially sleep) but can come awake more quickly than C3. I did read somewhere that C3 can cause problems in some scenarios (not specifically related to OPNsense).

htcp congestion control is fairly not relevant but is used in software routing, you can leave default of cubic in-place. I just changed mine, htcp docs do show a few perks over cubic, but I think you only really get to exploit any advantages in systems that can route much faster, perhaps 10Gb+. To use htcp, you need to load the KLM that provides the functionality before telling system to use htcp. There's a good handful of KLM's included in kernel build, many of them not loaded.

Hyperthreading off is by default (via OPNsense). I believe HT for the OPNsense workloads causes a performance hit. HT is good for some workloads, not all.

entropy mask
# this removes some time consuming resources from entropy for random

Random is better when entropy is better. Entropy can come from hardware devices, but it also uses resources. This mask takes out a couple more items from entropy harvesting, which means more system resources available for OPNsense.

You can document existing harvest settings, just find the random settings from sysctl -a

Another speed test I have done, did a Netflix speed test on a wired PC, it was getting just shy of my ISP rate. From the fw itself (download files testing) it gets just about my ISP rate. This however is just a single client test. My next testing is using multiple wired clients.


also see https://medium.com/@truvis.thornton/opnsense-firewall-configuration-performance-tuning-for-multi-gigabit-internet-and-better-speeds-in-cfc80c49c544