SFP+ to RJ45 slow WAN speeds?

[viper359]

Looking for some advice from you IT hardware professionals and labbers
I am having issues with WAN speeds after a few weeks, it slows to under 1Gbps
ISP Modem built in speed test shows full allocated speeds of 8Gbps up and down
Yes, I have rebooted modem anyways, just to be sure
Zero info in logs. Speed test from CLI also confirms less than 1Gbps speeds
All links remain showing at 10gbe
If appliance is powered down for 5 minutes or so, full PPPOE speeds return
Sometimes a few reboots in a row will also fix the issue
so, I switched back to SOPHOS OS for the Sophos appliance, thinking maybe a FREEBSD driver issue?
However, its now happened running this OS, which is Linux based.
Complete network info below my post

I am thinking maybe the SFP+ to RJ45 module could be the issue, but, would that make sense? Do these things just sorta kinda work, but not fully work? The SFP module is hot, but I have been told that's normal for these types of SFPs.

The only other thing I can think of, is the SFP port on the appliance itself is the issue, but, if it was, I would assume link speeds would change or at least something showing up in logs, of at least its maker, SOPHOS. Nothing ever seen.

INFO BELOW
Appliance: SOPHOS XG450
ISP: Bell Canada
Speed profile: 8Gbps symmetrical
Connected to 10Gbps ISP modem WAN port via RJ45 to SFP+ converter in top SFP+ slot of Sophos appliance
LAN connected by SFP+ to RJ45 module to 10Gbps Unifi switch
10Gbe switch to 1st floor 10gbe dumb switch, connected at 10Gbps to a Asus BT10 wifi system
2nd floor wired from 1st floor 10gbe ASUS BT10 WIFI.
Full 10gbe speeds throughout the LAN from any 10gbe capable device

Any thoughts would be greatly appreciated. i would like to switch back to OPNSense, but I gotta figure this out

[Seimus]

If appliance is powered down for 5 minutes or so, full PPPOE speeds return
Sometimes a few reboots in a row will also fix the issue

You mean the modem or the OPN e.g the hardware on which its running?

Full 10gbe speeds throughout the LAN from any 10gbe capable device

So basically when you have that issue on WAN, LAN is performing OK? So its related and isolated only to the WAN.

Could be indeed a thermal issue. Too high a temp has direct impact on performance.

Regards,
S.

[viper359]

Hi
Thanks for your response. Yes the sophos appliance. If I power it off for a few minutes, pppoe speeds resume. A few reboots in a row has also worked for both OPN and Sophos OS

Yes the LAN always works at full 10GBE speeds throughout. Which is also using the same type of sfp+ to rj45
I will see if I have a spare sfp I can try

[pfry]

[...]If appliance is powered down for 5 minutes or so, full PPPOE speeds return[...]

That does sound like it's cooling off, but if that's the case, it should be consistent. Do you have any other connectivity options? e.g. a DAC cable and a switch with 10GBASE-T and SFP+ ports.

[Seimus]

Definitely try to replace the SFP, potentially some that could thermally perform better. Copper SPFs+ are known to be pretty hot..

SFPs have often a MON, a capability to measure temp. Not sure if yours has or if its readable by the OS but you could try to digg it out if you can see what are the temps during the issue.

Maybe you could as well if you are not afraid, open the device and check how is the SPF socket cooled, I mean is there a thermal pad maybe?
As well where is your FW placed? Is it in a rack or in a open space? If possible try to put it in open space.

Regards,
S.

[pfry]

[...]SFPs have often a MON[...]

Grr. Good point:

Code Select Expand

root@fw:/home/user # ifconfig -v ixl3
ixl3: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: x710p3 (opt4)
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether 3c:fd:fe:e7:2d:8b
        media: Ethernet autoselect (10Gbase-SR <full-duplex>)
        status: active
        nd6 options=9<PERFORMNUD,IFDISABLED>
        drivername: ixl3
        plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
        vendor: Intel Corp PN: AFBR-709DMZ-IN3 SN: AA202830LM3 DATE: 2020-07-11
        module temperature: 27.90 C voltage: 3.35 Volts
        lane 1: RX power: 0.51 mW (-2.92 dBm) TX bias: 5.46 mA
root@fw:/home/user #

[nero355]

Code Select Expand

module temperature: 27.90 C voltage: 3.35 Volts

That doesn't seem to be that bad : Those things can go to 50/60 degrees Celsius easily!

Some other things you should know :
- It wouldn't be the first time a Ubiquiti UniFi Switch has issues with buffering 10 Gbps traffic...
- They also do not like Shielded CAT5e/CAT6a cables !!
- SFP+ to RJ45 modules like to have 2 slots next to them free to avoid overheating !!

Good luck! :)