I did transcoding with MediaCoder which supports GPU video encoding. Tested it on a 1 hour MPEG2 HD video (720 x 576, 3.2 GB) and set the output bitrate to 8000 kbps. The transcoding time is surprisingly only 14 minutes (the stat showed 220 fps) with CPU utilization of about 60%. I guess the CPU is partially used for decoding.
Tested Badaboom 1.2.1 on the same video file, it lasted 18:30 minutes (90 fps, 8000 kbps) with CPU utilization as low as 10%.
I didn't do any quality comparison between the outputs, but in the first glance they look similar.
Based on this experience, I think GPU may have give us something in terms of additional computing power. Although I can see that it may take a while until we can fully utilize it. But again it can flourish only in application where the computing can be paralelized.
Note: The sytem for testing (i7 940, GTX 295, XP Pro 32 bit)