While getting all my parts into my new case, i realised that i needed to take the front panel connector for the USB 3.0 case ports out of the motherboard to fit my graphics card. While taking it out i had a lot of trouble and the bracket that is on the board around the pins came off. If i plug the connector into the pins they still work fine but should i worry about it?