So you have installed OPN on that physical server, not a Virtual machine inside it, right? Just to be clear on variables.Then you need to have a better baseline. The testing through it is from a VM on another hypervisor. Check with another but otherwise make sure is not what is throwing you off course. Unlikely since it was 900 mbps before, but worth double checking.Then back on OPN. Tunables are in System > Settings > Tunables. But don't change anything yet, until you know what you want to change in a controlled way.What do you get from "ifconfig" ? You can hide the public ip before posting it here. It'll be interesting to see the driver in use and the media identified.
What I meant by multiple threads is the '-P' option on iperf (if that is what you use) which you absolutely need to measure speeds above gigabit. A single TCP connection will not be sufficient to test at those speeds.RSS is in tuneables, but depends on actual driver support for your NICs and the syntax depends on that. I do not use Broadcom, so I cannot help with that.There may be other driver-level settings for your NIC that affect buffer memory, queue sizes and others which come into mind when there are problems on the receiving end (one prominent example would be flow control, which could severely impact receive speed).P.S.: You actually have rxpause and txpause enabled. That might be it. I do not know how to disable flow control on Broadcom under FreeBSD. Intel uses tuneables: https://lists.freebsd.org/pipermail/freebsd-net/2012-July/032868.html
You are on the verge IMHO of having to investigate tunables for your nic's driver:https://man.freebsd.org/cgi/man.cgi?query=bnxtI'm not saying go changing things yet, until you are sarisfied the out-of-the-box settings are not ideal.Check with mimugmail's notes on how he tests for performance https://www.routerperformance.net/opnsense/opnsense-performance-20-1-8/
What I meant by multiple threads is the '-P' option on iperf (if that is what you use) which you absolutely need to measure speeds above gigabit. A single TCP connection will not be sufficient to test at those speeds.