Slow upload with timeouts

[akito]

Hello,

I'm having an issue with OPNSense that I can't wrap my head around regarding very low upload speed.
I'm running OPNSense 24.1.6 in a Proxmox KVM. All packages and updates are installed.
Hardware CRC, Hardware TSO and Hardware LRO have been disabled on my WAN. This I did as a suggestion on another site trying to get rid of the slow upload.
I have no VLANs installed.
I have tried using both the onboard NIC as well a USB-NIC which I had passed through to OPNSense.
My internet is 1Gb up and down. Download is going fine at around 900Mb/s. Upload however starts a roughly 2Mb/s but will quickly drop to less than 10KBb/s which often results in time outs of uploads or webpages don't load incompletely.
Intrusion detection is disabled, I have no Suricata or something else like that installed.

Anyone's got any idea what might possibly be going on here?