Speed throughput

[opnbob]

My ISP provides service at 1Gb. I have tested with my laptop connected to their modem and speed reports are in the 1Gb range. Great! Enter OPNsense (23.7. the mimugmail repository provides a speedtest plugin. That plugin is reporting 200Mbps. When I run 'speedtest-cli' from the commend line I am getting 108.63 Mbit/s. Clearly tuning is required to improve this speed.

My OPNsense is running on a Protectli box (FW4) with an Intel(R) Celeron(R) CPU J3060 @ 1.60GHz (2 cores, 2 threads). Searching the web yields little to nothing to assist a BSD novice like myself. (Years of Linux experience and I have a touch of BSD experience however not enough for this issue I am tackling.) This is a home network not commercial. But please understand that the bottleneck appears to be OPNsense. I found this, https://binaryimpulse.com/2022/11/opnsense-performance-tuning-for-multi-gigabit-internet/ but stopped when he began discussing a 24-core cpu. And it is an article from 2022. Since then, there has been several OPNsense version upgrades. So far, I have not found any helpful info that is at home network or SOHO network level of use. Where can I find the info I need to improve my ISP speed throughput?

[meyergru]

Depending on how you connect and what you want, a J3060 might not be enough for that. FreeBSD is notorious for its abysmal PPPoE performance, so if you need that, it will be a problem.

Plugins like suricata and zenarmor will be even more strenuous on the CPU. Also, if the speedtest is executed on the firewall itself, it puts more load on it. You may get more comparable results if you measure traffic through your firewall than if you measure it on your firewall.

Other than that, you could try receive side scaling (RSS).

[Edwin70]

I have a similar Protectli, the FW4B, Intel J3160; Quad Core 1.6GHz with 4 network ports. From my cable provider I get 1 Gb down and 120 Mb up. And in practice I get 940 Mb down and 120 up. Maybe the quad core and the extra memory helps. But I also don’t run stuff like suricata or zenarmor. Just plain firewall with some extra rules and Unbound dns.

[meyergru]

Do you use PPPoE or plain DHCP?

[sja1440]

I too have a Protectli FW4B.  I have a 1000/300 line.  My ISP requires me to use pppoe over a vlan when connecting to the ISP supplied ONT.

My FW4B did not have sufficient cpu power to manage the bandwidth. In order to order to understand what sort of device I would need, I carried a few informal tests:

• FW4B + OPNsense got around 750 Mbps 
• FW4B + openwrt got around 780 Mbps
• FW4B + IPFire got around 550 Mbps
• ISP supplied router got around 780 Mbps 

For the tests OPNsense was always configured in a minimal configuration with the Protectli suggested pppoe optimisations, no IDS/IPS, no OPNsense netflow, and no QoS. Powerd was always enabled. No other devices were connected.

Observation showed that OPNSense with the FW4B is cpu bound,

I decided to get a massively over spec'd VP4670 and with this I comfortably get 940Mbps down with IPS enabled. Back of the envelope calculations suggest that it should be able to manage at least around 2 Gbps - but of course I havent tested that.

[Edwin70]

Do you use PPPoE or plain DHCP?

Just plain DHCP. Sorry, probably an important detail.

[opnbob]

Welcome all. Glad to see this discussion.

I have moved back to my Protectli FW4A-0 running an Intel(R) Atom(TM) CPU E3845 @ 1.91GHz (4 cores, 4 threads) with 8Gb RAM. My WAN connection is DHCP from the ISP modem (that is default setup as a bridge.) Currently I have 802 Mbps down. (Not to concerned with my up speed of 39.6 Mbps.)

I can nbow see that my issue was hardware related and moving back to the FW4A-0 has resolved my dissatifaction.

Thanks erveybody. Your thought helped me. (I consider my part of this decision closed.)