The advantages of Windows Home Server is that it is a little easier to setup for a Windows only environment. Downsides is it costs money. I've never really used it so I don't know of any other benefits.
*nix based systems are obviously free, and might be a bit harder to setup. However, they run better on older hardware and can easily be setup for a mixed OS environment.
I setup my file/backup server with Ubuntu Server 10.10 (upgraded to 11.04 just recently). My decision to use Ubuntu was based on the fact that I have limited linux experience and wanted something easy, with a large user base to ask questions. I also figured that I could use my server for other purposes. I used Webmin as my interface and while convenient it doesn't make...