The CPU itself is tiny, Said small silicon chip still needs to have all those connections to motherboard, which take space.
The heat spreader (the metal thing on top you attach cooler to) is actually almost 9 times bigger than the silicon chip itself.
The silicon chip containing said CPU and other things sits in the middle and there are no connectors directly beneath it to give more room for routing the connections to the socket.
1150 or so of them after all..
That is in no way official answer, it is my own take on the matter.
Edit: as a side note, I7 haswell (like 4770) was 177mm2 which means 13.3x13.3mm