GPUs do get more efficient over time. However, rather than delivering the same performance at lower power, they exploit those efficiencies by delivering better performance at the same (or even greater) power!
What's worse is that they run these devices well above their peak-efficiency point, chasing ever diminishing returns for the ability to claim the highest performance.
Basically, if a GPU maker can deliver X performance by making a bigger die or by increasing clock speeds, they'll go with the latter option, because that costs them much less. The downside is that it burns more power.
We're seeing a similar story with CPUs. In both cases, if customers only cared about efficiency, then perf/$ would suffer.