A quick chime-in -- I doubt buffer bloat is having that significant of an effect. Keep digging. There must be a major bottleneck somewhere.
You didn't inadvertently set up a 100Mbps interface speed by accident? Running over WiFi?
I recently had a major speed issue when using wireguard, turns out that I didn't have my hardware accel. features enabled in System->Settings->Misc.