Sounds to me like you discharge yourself to the PC, or you see sparks when plugging something in, which are both completely normal and safe.
Now, electricity is dangerous, so don't conduct weird experiments on your own ok?
And, not all homes are grounded. The only grounded outlets in my home are the kitchen and bathroom, not in any other room. Never had a problem with electronics ever.
Though if that's the case, don't touch the case and something that's grounded, like your radiator.
All the best!