This is a common problem that occurs with amplifiers powered by switching power supplies. The switching frequency used in high power sources such as used in pc are close to the threshold of audible frequencies of 20KHz, and therefore cause interference in the audio. To work around this problem quality sources have common mode filter, but not always the plugs of these sources are grounded and so these filters do not work. Therefore, it is necessary that the sources are effectively grounded. If this filter included not solve the problem you need to add another Butterworth filter of at least third order, consisting of a capacitor (C = 47uF), an inductor choke (L = 10mH) and a capacitor (C = 100uF), connected in series in the supply of 5VDC USB. I advise that you use a USB extension cable to mount this filter so as not to spoil the characteristics of the source.
Another ease solution is the use of a mobile charger with USB output to power this circuit. The low power charger is an AC / DC converter that uses switching frequencies much higher than the high-power sources, and thus does not cause noise. Just watch out how much power your device needs. The power device should not be greater than that delivered by the chargers. Most chargers provide up to 1A output, while the Razer headphones utilize maximum 0.2A.