Very simply put, overclocking running a component at a faster speed than its rated frequency. The most commonly over clocked parts are the CPU, the GPU and memory.
Your memory and GPU will be the easiest to overclock. Overclocking your CPU will require a good after market cooler. You never want to overclock your CPU with the cooler that comes with it. Bad things can happen.
As far as HOW to overclock your system. There are many fine guides on the web and here on Tom's Hardware on how to accomplish this. It really depends on what CPU and motherboard you have. Some parts are more able to be over clocked than others.