Welcome to Tom's Hardware Forums,
In some cases, the front panel connectors are simply hot melt glued into place. Carefully cutting the glue with a sharp knife should free it from the case. Other cases it's screwed into place and yet other cases use a combination of screws and glue. It is fairly straight-forward of a task.
I should probably point out that most people simply bundle up the unused panel wires and tuck them away - you may tire of the HAF-X and want to sell it someday
Hope it helps