its not slower just 4 sticks will take more power than two
meaning if you overclock you need to pump more voltage into to your ram (4 stick) to get a decent overclock
so go with two sticks since its easier to overclock.
hope that helps
In my experience 2 sticks is faster then 4.
There are also many stories going around about poor performance on high end rigs and most are fixed when the owner replaced his 4 sticks of ram with 2 sticks.
This may be completely false i dont know but its something to consider.
( You can also upgrade with more ram later without having to replace sticks if you go with 2x2gigs )