How much? is really dependant on the individual chip, some clock better on stock voltage and some clock badly and need a large voltage bump.
However you could probably push into 4ghz-4.2ghz up to 4.4ghz (if you're lucky) without bumping up the voltage. I have not seen anyone get above ~4.5ghz without a voltage bump.