Speed drop at max speed

[BiTRiP]

Hello,

I have a N100 mini router with 4x 2.5Gb ports. This box has 16GB memory and runs OPNSense natively on its SSD.
The ISP (KPN Fiber) uplink is 1000/1000 Mbit/s and is directly connected, so no modem in between.

When I do a speed test the download speed goes right to around 930Mbit but then immediately drops to around 600Mbit.
The upload test also goes to this 930Mbit but stays around this speed and looks fine.

When I replace the OPNSense box with the providers modem then the speedtest remains stable at 930 for both down- and upload.

Any idea why I have this problem?

At max download the CPU is just having 50% load and memory is also fine.