Slow Download Speed

[Lruts]

OPNsense 24.1.1-amd64 connected directly (no modem/router) hosted on a Protectli Vault FW6A.
There is no special configuration besides DHCP on IPV4 and IPV6 disable.
I have 1 GB service with Frontier, but tests ran by Speedtest plugin are showing downloads of 350 Mbps and uploads of 900 Mbps consistently.
What is weird is that every time I restart the device, the first two or three tests will show downloads of 900 MBps but after 30 seconds it will go to an average 350sh Mbps, the uploads will remain above 900 Mbps.
    Average Download speed:    429.98 Mbps (min: 239.03 Mbps, max: 940.48 Mbps)
    Average Upload speed:   917.12 Mbps (min: 854.17 Mbps, max: 940.77 Mbps)
    Average Latency:       6.99 ms (min: 6.07 ms, max: 7.42 ms)


Any pointers will be greatly appreciated.

[johnmcallister]

Do you have a baseline of earlier tests done on the same hardware & same connection, using older versions of OPNsense?

If yes, consider summarizing those tests here.
If no, try using a series of other speed testing providers for comparison. (Fast.com is one example, there are several others.)

It's possible that Frontier provides 'burstable' 900 Mbps downstream but after repeated bursts from a single host or within a certain time-frame they rate-limit your downstream connection temporarily.

[Lruts]

Unfortunately, I don't have a history of tests, I just started today after several days of lagging.
It doesn't seem to me that Frontier is limiting the downloads, as I still get 350 Mbps after waiting for an hour, but as soon as I reboot Opnsense it goes to 900 for a short time.
I am suspecting that something is getting loaded that is limiting to around 375 Mbps as I can't go beyond that on all further tests.

[johnmcallister]

I would perhaps approach this by using a separate local bare-metal device, connected directly or through a reliable and otherwise-unloaded GB or 10GB-capable switch, to test Opnsense's max bandwidth. (Using iperf & similar tools.)

You might be able to reproduce the issue with your local test setup, which would rule out anything to do with the ISP.

Or, if you can't reproduce it locally that might tend to point back towards something specific to the Frontier connection.

[Allistaire]

Unfortunately, I don't have a history of tests, I just started today after several days of lagging.
It doesn't seem to me that Frontier is limiting the downloads, as I still get 350 Mbps after waiting for an hour, but as soon as I reboot Opnsense it goes to 900 for a short time.
I am suspecting that something is getting loaded that is limiting to around 375 Mbps as I can't go beyond that on all further tests.

After Upgrading to 24.1 it was fine for a while, then I noticed intermittent speed problems. Especially on Download speed, but not limited to it.  When running speed test My download speed would be 30Mb but the OpnSense router would show 120Mb (speed of my line), I eventually downgraded to 23.7 and the problem is gone.

[Lruts]

After some testing, I found that the IPS mode (under Services: Intrusion Detection: Administration) was the one causing the problem.
Once disabled the test results (download/upload) are consistent (900 Mbps)