If you get a problem like you are describing.
It can point to a bad earthing of your motherboard, and the case chassis.
The case is always earthed via your power supply unit fitted to your case.
So it will or would point to a fact you have either a bad ground or earthing point of the power supply it`s self.
Or due to whatever brand of power supply you have in your system if of a cheap brand name.
It has a very poor wiring of the ground or earthing wire.
The ground or earth depending on what country you live in differs.
for example in the uk we have three pin plugs where one is the earth or to ground lead.
Where as in other parts of the world the plugs are two pins, because the voltage supplied is about 110v vs the 240v wall supply common in the uk.
If you ive in the uk you should check your plug and the wiring to make sure that the earth wire is connected or properly screwed down of the plugs terminal, if not a sealed or molded plug.
For the most part if you live in the US it suggests that something in the power supply is not right. or as said is a low cost power supply.