There would be no damage to your other components, if your card is only defect. However, if your card is experiencing a problem with power connection, or your card gets burnt, i would keep it put of my computer to prevent damaging the motherboard.
Also, i can't really see the point of having a broken graphics card taking up a PCI-E slot. There are only disadvantages by doing this