Most new motherboard now (especially P35 and Nvidia 6 series) support 8Gb of RAM. The issue is usually not with the mainboard but with the OS. I have 4gb RAM installed in my Asus P5B-Deluxe no issues, however, XP SP2 on detects 3Gb and will not map anymore. This is a limitation on the 32Bit instruction. The best work around for this although ive never had the chance to see this working is to goto a 64Bit OS or Vista either Buisness/Ultimate as other vista versions (i believe) also dont support more than 4Gb.
As for memory modules the less the better so if you can get 4gb in 2 sticks is better than 4gb on 4sticks.
RAID 5 is supported by higher end mainboard these days also, although when i build a RAID 5 for myself i got an addon card to do it. This helps with speed, scalability and doesnt rely on the mainboard but rather a dedicated card.
The system that i recently build that would suit you was a Q6600, with 2gb Patriot at 1066Mhz, Asus P5K-E (supports 8Gb), Adaptec 3805 SAS/SATA raid controller, with a 7300Gs (cheapy), 5x500Gb Western digitals in RAID 5.
Hope this helps