Ping: old school term is Packet Internet Groping.
What is it? It is a small packet of information that is sent across a network, or internet, to a destination and a response (or lack there of) is returned.
When accessing the internet with a ping, it must go across multiple routers and/or DNS servers. The higher the ping, the more likely the ping is to travel either more hops (router to router) and/or over heavily congested communication lines.
In reference to gaming:
The higher your pig, the longest it takes for your game packets to get back to. In gaming, you're actually uploading far more than you're downloading during game play. The "world" of the game is already known to your computer, the variable being world changes (building being destroyed, or marks left over, etc which would be downloaded) and your game movements (uploading these movements). Ping is a quick reference for lag in a game. If you had a high ping against someone with a low ping, the difference is they receiving and updating game information quicker. While a game may play perfectly fine away from others (you're in your own world, not synchronizing with anyone else), when you encounter another person, both systems must synchronize. The person who is able to do that quicker will generally have the edge. This is why you hear people in a game at a key moment say they 'lagged.' What happened is the other person was updating their game movement (or commands) and you were trying to download theirs while uploading yours. If your ping is higher, it takes just a little bit longer. In the grand scheme, that can relate to a fraction of a second delay in a game at a key moment.
In larger games like an MMO, when everyone gets together it starts to lag because now you have dozens, if not more, all updating their movements together and that has to synchronized and sent out to everyone. Often, game collision is turned off in large scale games because think about it.. multiple objects trying to take over the same space will be a nightmare to figure out who wins.
So in the end, ping is a low level, small packet of information that can provide a quick snapshot of what's going on with your computer. If you're hosting a game server, your ping is generally 1-2ms. If you're playing on a server in SanFran while you live in Atlanta, data has more hops to go through and that will eventually cause some congestion and thereby slow down your data by a few milliseconds. If you played on a local server in Atlanta, your ping will likely be good because you have minimal networks to traverse for your data. The less your data has to travel over the internet (the internet is a bunch of inter-connected networks, thus the term "internet") the better your ping should be.