I've seen similar situations where a link negotiates at full speed and looks healthy until sustained traffic exposes the problem. Your point about checking the PHY compatibility instead of only replacing cables or tweaking OPNsense settings is probably the biggest takeaway here. It also explains why changing software versions had no effect. Hardware interoperability at 10GBASE-T can still be surprisingly inconsistent, even when every component is technically supported.
"