Unless your temps are pushing 100C due to a poorly ventilated system, absolutely return it. Any decent brand 7770 should be very solid even at their OCs.
Of course you could overvolt it, potentially making the OC stable, but this will void your warranty and you'll be unable to return it.