I found a solution. It doesn't fix the fact that Steam generates globe images for certain games. However, I was able to replace the globes with the right icons.
- Right click on your globe icon.
- Hit properties, change icon, then browse.
- Find where your "steamapps" folder is installed and click on it.
- Go to "common"
- Click the specific game title (ex. Far Cry 4)
- Now find the folder that houses the icon. For Far Cry 4, click the "bin" folder.
- Find the word "Icon" under the "Type" column. If you look all the way to the left you'll see the correct icon that the game should be using.
- Click that Icon and hit "Open"
- Click "Apply", then "OK"
Your icon will now be correct with no more lousy looking globe.
**For some games you'll have to click on something other than the "Icon" type. Basically anywhere that you see the icon for your game all the way to the left, will work. For Borderlands 2 I had to click a different one but still got it to work**