I'm going to assume you're using Windows 7 when going through the troubleshooting steps, although all Windows versions (at least the ones that people still use) should have these options and menus. If you're using OSX or a Linux-based system it would be a different process, although a similar one.
When the dongle is plugged in, go into device manager and see if the computer recognizes that there is another sound device. It may show up as no drivers are installed. If it shows up in there, go into your playback devices and make sure the the Logitech hardware is selected as default instead of the onboard audio (or whatever audio solution you're using).
If this doesn't work then I would suggest installing the drivers for the headset off of Logitech's website. My understanding is that the headset is still supposed to work without the software or drivers, but with my experience with the headset (along with the experience of all of my friends that own this headset) is that it is buggy. Sometimes very buggy.
Hopefully this will work, but if it doesn't then I would suggest getting it replaced, whether that is through the retailer that you bought it through or through Logitech. Logitech has very good customer support and they will get you hooked up as long as the headset is still under warranty.