A site I used to follow, back in the early 2000's, was gpgpu.org. Here's a snapshot from the Internet Archive, to stoke your nostalgia:GPGPU was what was making the rounds in all of the textbooks in the mid-2000s. General Purpose Graphics Processing Unit. Since the idea was you could now run non-graphics code on them, which was 'new' at the time.
At a job I took, back in 2002, they even got me an ATI 9700 Pro, with the idea that we could program it to accelerate some of the computational work we were doing. It never reached that point, but I think its GPU was the first to support floating point shader programs (using a special 24-bit number format). The kinds of shader programs it could run were extremely limited, both in size and complexity, but it was still enough to make it interesting for a wide range of applications.
The last thing I'll say about that card was that its fan failed after a year of low-intensity use. ATI was at least easy to deal with, regarding the warranty process, and cross-shipped me a replacement.