If your adapter connects from a TRRS type jack to 2 x TRS jacks. You may still be sharing a ground connection. It is still worth checking if the noise goes away when using just the mic with the headset connected.
With a TRS you have Ground, Right and Left with TRS you have Mic, Ground, Right and Left(a shared ground)
You can test this with sound recorder on your computer.
Normally these devices are expected to be used with the same sound card for output(headphones) and input(mic). You simply are now setup this way.