If it was particularly difficult then people like me would not be able to do it. There are a few essential points to bear in mind about the process. From my own perspective it takes about 2 hours to piece together a complex build. Since you are thinking about creating your first machine make sure you set aside more time so that you don't get interrupted and lose track of what you're doing.
As far as piecing things together goes, most components have shaped connectors so they can only go one way, CPU's generally have a few indicators on them so that you know what way to insert them, Intel CPU's have a pair of notches cut into the PCB package that serve as an installation guide. RAM has a key notch that's about 45% of the way along the PCB to ensure that you don't insert it the wrong way and power connectors also have shaped connectors to prevent you from inserting one into an inappropriate socket. Though I will point out that you can force power connectors home so check them thoroughly.
But really the key to building a system is getting the component selection right. certain CPU's only work with certain types of Motherboard, some motherboards only function with specific types of RAM. Some systems aren't particularly expandable while others are more so, certain cases will only accept a motherboard of a specific size.
So my advice is to write down a specification, ensure that you 'marry' technology properly (for example if you want to use an Intel CPU (make sure you select a motherboard with the correct socket and chipset to support it). DO NOT SKIMP ON THE POWER SUPPLY UNIT, I can not count the number of systems that I have dealt with that have fallen apart because the PSU was a no name model that was not up to the task!!! While you do not need to purchase the greatest available motherboard on the market it represents the potential of your computer, your capacity to upgrade the system you build is limited by the motherboard and the PSU.
Anyway, if you decide to do this I wish you luck, but start off with a pen and some paper and write down the specification of the machine you want. Do some research on each product you choose to ensure that its the right fit.