Separate headphones and microphone - correct?
When a microphone is "off" and still can be heard then that microphone is not really off. Electricity is flowing where it should not be flowing and that leads me to believe that there could be a short circuit.
Being "a $2 China special" suggests that the microphone could be faulty or failing. One immediate thought being a faulty on/off switch. You get what you pay for.
However, because that particular microphone works fine on other devices then the problem (a possible short) is in your computer or perhaps the headphones if the microphone is connected there.
Likely a damaged port, microphone plug or cable. Maybe just a bit of physical misalignment. Does not take much to cause such problems.
And adapters, if any, can go bad as well. What connection path are you using?
Does the problem occur with the other two microphones? If so then that is even more indication that the problem is within your computer.
= = = =
That all said, there is no need to be rude or insulting. Continuing such behavior is unnecessary and has consequences. Please read the Forum Rules.
We are all volunteers here and the troubleshooting process has to start somewhere and usually starts with some initial questions.