[SOLVED] what is wrong with my new gpu

Solution
Looks like some VRAM corruption issue, mostly because of a faulty GPU. You card might be having some problem. RMA it, assuming it is still under Warranty, and it is dead.