Recording or mixing? Because if you're mixing then it's more on the headphones, DAC and AMP.
If you're recording, your most important parts are going to be your fans and coolers, underclocking your parts so they don't need to run active fans.
What sort of budget are we talking about?
If you're concerned about mechanical noise and have enough budget, just keep the computer in a soundproofed room. If you're concerned about digital noise you'll want a high end motherboard and expensive noise isolating components.