Yes, overclocking the RAM by jacking up the voltage can allow you to run at faster bus speeds and/or faster timings; as you note, it depends on the quality of the RAM. Certainly, not all DIMMs with 1.8-1.9V specs are good quality RAM. However, RAM that needs to be overclocked to reach CL4 at DDR2-800 is certainly NOT good-quality RAM. Similarly, RAM that needs to run at 2.2V or higher to run at DDR2-1066 is certainly NOT good-quality RAM. There was a time not too long ago that one could buy DDR2-800 CL4 RAM spec'd at 1.9V. Similarly, you used to be able to buy DDR2-1066 RAM spec'd at no more than 2.0V.
The stuff being sold these days, however, is just average DDR2-800 jacked up to ridiculously high voltages like 2.2V. The reason for this is that 2x1GB of real good-quality RAM would cost you $200-300 and nobody is willing to pay that when you can get decent DDR2-800 CL5 RAM for $40. Thus, the manufacturers just take the DDR2-800 CL5 stuff and overclock it, misleading buyers by listing an above-standard spec voltage in the "fine print".
Again, my question is: if the overclocked stuff being sold by manufacturers is really good quality, why aren't they willing to reveal its speed/timings under standard 1.8V/1.9V settings? When someone doesn't want to reveal standard info about a product, doesn't that make you suspicious?