In my opinion, the best place to start is with an old system that you can take apart and put back together. If you don't have one of those, take $25-$50 to a local pawn/thrift shop and get one. Familiarize yourself with what components make up a working system:
Case, Power Supply, Motherboard, CPU, System RAM, Hard Disc Drive, Optical Drive, Graphics*, Networking*, Audio*, and the Operating System.
Open the case and get to know each of these components.
* Note: in many systems, the motherboard is capable of handling the graphics, networking, and audio of a computer. If the motherboard is controlling these functions, the connected ports will be on the motherboard's rear Input/Output panel. These functions of a computer *can* be upgraded with discrete cards.
Next, find out what motherboard is installed in the computer and then download the manual for said board from the Internet. Read it, read it, and then read it again. The motherboard is like the Central Nervous System of the computer. The manual will tell you how each component connects and interacts with the motherboard and all the other components.
Once you understand that, you need to figure out how components have evolved over time (generally just understanding what the current technology is) and then, finally, what components you need to fulfill the task requirements of the PC.
-Wolf sends