DEC750 realistic 10G expectations

[ou1]

Hello community

I have a DEC750, a 2-year old model with 2.5G ports.

On the WAN side, I have a 10G XGS-PON module, 10G internet plan from my provider. The WAN interface requires a non-default VLAN tag.

For testing, I plugged my PC directly into a 10GBe transceiver in the DEC750 using default VLAN 1.

Doing internet speedtests, the highest speed I can achieve is 5.2Gbps (iperf3 with parallel streams, speedtest net, cnlab). Is this to be expected? I don't want to spend too much time trying to increase this if it's already maxing out the firewall throughput of the DEC750. What I haven't tried is disabling the firewall on the DEC750, but this falls into the "spending too much time" category since I'd never use it in that configuration.

Using the router provided by my ISP, I can reach just over 8Gbps in speed tests.

[OPNenthu]

I don't own one but the published port-to-port throughput for DEC750v1 and v2 (as tested with TCP, full-duplex, unknown # of streams) is 8.5Gbps, so for a pure firewall application without IDS/IDP I don't see why you couldn't get similar results.

Aside from that, I have doubts about the SFP+ modules and VLAN setup you mentioned.  A lot of variables there.

1) Is it reasonable to compare your XGS-PON module to the ISP modem and expect similar results?  Have you been able to separately test and verify the module on another router box and confirm that it could hit 8Gbps with your ISP?  (I'm not saying it doesn't- just that we can't assume.)

2) Do both of the transceivers negotiate properly?

3) Do you need any specific NIC and VLAN offload settings or tunables (e.g. RSS) that are recommended for the DEC750?  Maybe your WAN connection is also impacted by some overhead due to VLAN tagging.

4) AFAIK, there's no "default" VLAN in OPNsense.  Did you explicitly create one with VLAN ID 1?  Not that it matters really, but could be some VLAN filtering overhead issue as well (?)

--

Update: Just in case...

https://docs.opnsense.org/hardware/defaults.html

Looking trough the default config, I don't see any interface offload settings or tunables that aren't already default in OPNsense, except for maybe the ones related to Meltdown & Spectre mitigation.  However there have been threads here recommending to at least enable RSS.

https://forum.opnsense.org/index.php?topic=24409.0
https://docs.opnsense.org/troubleshooting/performance.html#receive-side-scaling

--

UPDATE 2:

Interesting recent thread here: https://forum.opnsense.org/index.php?topic=49030.0

Presumably both of your SFP+ modules are using the same speed and not mixing (you'll need to check) but I wonder if maybe you are hitting a thermal limit with XGS-PON on the DEC750.