Partially it is the load on the steam servers. This is hard to predict because they have multiple data centers and you have little control over which they download from. When the servers are busy they seem to degrade people who are downloading at high speed much more than the lower speed users.
In addition it depends on the game and how it is distributed. Some games it will download multiple files rather than 1 larger file. Lucky they don't download the many thousands of tiny files games have individually. Still all those little files tends to be why my machine will download things fairly quickly and then take forever to install even on a SSD. Pretty bad when it takes more time to transfer disk to disk than it does to...