USB headsets only work on computers with compatible drivers. They do not work with stereos, MP4 players, etc.
3.5mm headsets are more compatible. They work with old tape decks and with my phone.
3.5mm headsets require that You have a working sound card in Your PC. USB headsets bring their own sound card with them.
If my existing hardware supported one but not the other, I'd go with the one my existing hardware supported. In my case, I have lots of aging audio gear, so I scoff at fancy new USB headsets, while drooling all over a used and abused 3.5mm headset.