New user with low upload speedtests

[thgardentool]

Hello.  Looking for advice on where I might be able to start the troubleshooting on much lower than expected upload speeds on various speed tests.  I'm on a symmetrical 1Gbps line through AT&T and only seeing at best ~300 mbps up at the firewall or any wired devices behind it.  My AT&T RG is able to still do its built in speedtest at its limit up and down.  The prior router would also allow its maximum around 860 up and down.  Download is fine and consistently seeing ~960mbps.

To try to give a bit more details.  The firewall is installed on a new J6413 box with 32GB RAM and a 64GB mSATA.  Right now I only have 2 of its 6 NIC in use one as WAN and one as LAN.  I'm not seeing any crazy CPU usage or when I did an SSH and checked LOG.  I do have Zenarmor installed and running as more information as well as ability to block certain things was one of the reasons to move away from a consumer router.  On that note though my desktop was still having low upload speeds prior to installing Zenarmor.  Otherwise I have the community repo installed to get Adguard Home which is installed but not running so far, and the only other thing was grabbing the speedtest off of it so I could do it from the dashboard instead of CLI.  Haven't gotten crazy with trying to do and learn firewall and routing rules either.

[cookiemonster]

What speeds do you record when you do your speedtest from a wired pc? Try a couple of speedtester sites as they tend to report differently from each other.

[thgardentool]

What speeds do you record when you do your speedtest from a wired pc? Try a couple of speedtester sites as they tend to report differently from each other.

Sorry I realized reading my original post was not very clear.  I was also doing tests on the desktop and was receiving the same results on the same servers that I was directly off the firewall.  I can get full expected speeds off of the AT&T RG, which for anybody not familiar acts similar to a modem, and is in front of the firewall.

I did keep working on and trying to research some of this over the weekend.  I came across a section in the docs on poor thoroughput and worked through most of it to no avail.  There was a suggestion to try a switch between the RG and the WAN interface that I have not tried yet.  I do have a spare switch I can pull out sometime to try when I can afford to have some internet downtime.  I also have not tried to change any ethernet cables, although the same cable was used successfully between the RG and the prior router so I don't anticipate it to be the problem.  I will change it out to verify.

There was a longer thread in a different sub-forum that at first glance I thought was regarding poor iperf results in VM environments.  A bit more reading through it seemed to apply a bit wider than that.  It was talking about testing some of the tunable settings so I will dive deeper with it soon.  One of the suggestions might be related to one of the software fixes for one of the Spectre vulnerabilities.

Additionally I did try some tests with Zenarmor in passive mode, with the emulated netmap driver, and services completely off.  None made a difference.  And as previously mentioned, I was getting slow speeds on the desktop prior to installing Zenarmor at all.

[cookiemonster]

Actually it was I who missed that you mentioned the wired devices behind the firewall, my apologies.
You are on the right track. I would keep any services that work the packets off the wire like Suricata, Zenarmor, etc. Those are the main ones. But from there is down to low level diagnosing. Things you can do is check the link is syncing at the right speed and duplex.

[thgardentool]

I found some time this morning that I could live with the reboots of changing around some of the different tunable settings I've seen suggested as possible fixes.  The two main groups were related to CPU vulnerabilities and CPU Core/Thread stuff.  Neither group of settings seemed to have done anything to fix my problem.  Also I have verified all the line speed and duplex are all reported correctly.

Where I'm at right now is still find time to be able to live with the longer downtime to try the switch between RG and OPNsense, replace Ethernet cable between RG and OPNsense, and all else fails may do a fresh install again just to make sure I didn't accidentally do something weird in the guided install that would be causing something funky.  But as far as I could tell never using OPNsense before is it was a pretty generic default installation.  And then after that if its still giving me issues, may look at some more advanced measures of moving and swapping things around to try a live boot environment off my desktop to see if it still has the same issues.

[chamley]

Did you fix this?  I have a similar problem with upload being slower than my previous router on a symmetric connection.

[thgardentool]

Did you fix this?  I have a similar problem with upload being slower than my previous router on a symmetric connection.

No I have not yet been able to figure it out.  I tried the last few steps from the troubleshooting doc involving swapping some various patch cables between the OPNsense box and the RG.  Tried the unmanaged switch between them.  Same exact results.  I am hopeful to get the downtime to do a complete reinstall to see if I borked something on the initial config.  Unfortunately with this particular box having i226 NICs I can't go test out pfSense CE either to try to gauge whether its a software thing or hardware performance.

For the moment though ~300Mbps upload is still quite a bit and do see close enough to a full 1Gbps down so it's not critical.

[thgardentool]

This week I threw on an install of FreeBSD 13.2 on my desktop on an unused SSD.  Installed iperf3 on both the OPNsense and the desktop machines, which was able to run at full line speed between them.  And the FreeBSD machine using the CLI Speedtest was able to hit max speeds utilizing the closest AT&T speedtest server.  The OPNsense machine using CLI still haven't seen a full speed test, but I do get higher than the mimugmail repo's GUI plugin version has ever done.  So I'm not quiet sure this is solved but I'll keep digging as I'm able to.