The issue you're having is called bufferbloat. It causes one or a few devices to be able to hog large amounts of bandwidth and causes your ping to go up. If you get a router/firmware that claims to support anti-bufferbloat features, pretty much all of what you have ever experienced as "the internet is slow" will go away. All without having to configure any priorities.
Of course it won't make you download faster, but it will make everything feel more responsive under load and more fairly distribute bandwidth.
OpenWRT claims to have anit-bufferbloat features.