Home Server, With a Twist (Brainstorming)

[Pyroflea]

It all began with reading an article, THIS one to be precise. I've always wanted to build my own case. I've seen plenty of other more complex builds, but something about this one just did it for me. I'm not sure what, but it's inspired me to start working on a project of my own.

Now I'm a student, who is moving this fall, so money is tight to say the least. This is all essentially just me rummaging through my head trying to pull a bunch of ideas into one material object, and I won't be starting on anything for some time. I'm just looking for some input as well as some suggestions as to what I should do.

Here's what I've decided so far:

- [strike]It has to run Linux (duh!).[/strike] The OS must be Open-Source. Most likely Ubuntu or some derivation of BSD.
- It has to be small. [strike]Either Pico-, Nano-, or Mini-ITX. I've found three boards, just trying to decide what features I NEED, and what would be nice to have. Most likely the Flex-ATX board from SuperMicro linked by linux_0.[/strike] Will utilize existing mATX rig.
- It has to be quiet. I don't want what will essentially be a noise box that I have to put up with 24/7.
- It has to be efficient. This isn't something I usually care about, but less power = less heat, and I figure it will make things more interesting.
- It must have A LOT of storage. I have a lot of junk, and I need a place for all that junk to be held. There's only gonna be more and more of it as time goes on.

- Zero peripherals. Must be accessible by SSH/WebMin.
- Will server as some of the following things. Not necessarily all, and not limited to the below listed. :
- DHCP/DNS Server
- Database Server
- Web Server
- Mail Server
- Subversion Server
- Backup Server
- iSCSI Target


Those are the "Must-Have's" for this project. Pretty simple, in my opinion. Now comes the fun stuff.

I'd like for this to be kind of an "Ultimate-at-home-network-tool", [strike]also serving as a Firewall, Proxy, Router, etc[/strike]. Not necessarily all those things, but you get the point. I want it to do everything possible. I'm not sure how feasible this part is. [strike] At the very least I'd like for it to serve as a Firewall, just because I'm paranoid and I like my security. I know that Linux can be used as a router, but if that doesn't seem appropriate for this project than just let me know and I'll just continue using my current router.[/strike] I don't know if a Proxy is worth my while, but if it is and can be hosted, then why not!

If you're starting to get my point (which you should be, I already told you earlier), I just want this thing to do all that it's capable of to have a secure, fast, convenient network.

I'm looking for a few things out of you guys.

1) What should it serve as? All those things, as well as anything I missed that might be beneficial would be fine with me. I'm open for anything.
[strike]2) What should I look into for the Operating System? Doesn't have to be specific, but a certain type of OS, or just some pointers would be awesome.[/strike]
[strike]2.5) Considering going with FreeBSD for this. Yay/Nay?[/strike]
3) What kind of software is available to do the things that I'm wanting (so that I can start what I feel will be an overwhelming amount of research).

Thanks to everybody who contributes, and I'm looking forward to hearing your feedback and suggestions.


- Jesse

***Updates to Post are in Italics***

 

[sub mesa]

Theoretically software RAID is superior without a doubt. This is because it has the most potential to achieve high performance. The hardware RAID generally does this quite well; but software RAID implementations vary from simple Windows-only onboard RAID drivers which are generally of low quality, to advanced Filesystem/RAID-engine hybrids like ZFS.

ZFS can do things no RAID can do: dynamic stripe sizes. For that to work, the RAID engine has to know things about the Filesystem, and vice-versa. So this can only work with a product like ZFS.

Regular software RAID options under Linux and FreeBSD are also of much higher quality than those available on Windows, where array splits and other trouble is quite common, even with the disks being fine.

So don't judge RAID0 or RAID5 - judge individual implementations of those RAID-levels. The RAID5 is just a spec on how to store data on disk; the driver will actually determine how much of the performance potential can be utilized and how well it scales.

 

[Pyroflea]

Awesome, thanks for the explanations. So I assume that ZFS is kind of "The Option" as far as software based RAID goes? How would this compare to the regular RAID options under Linux and FreeBSD that you mentioned? ZFS is compatible on both systems as well, is it not?

 

[sub mesa]

ZFS only runs well on FreeBSD and OpenSolaris at the moment; Linux has to do with the FUSE layer as ZFS and Linux license are incompatible to mix. However, there is another project which aims at porting an OpenSolaris module to make a real kernel-level implementation under Linux possible.

For the moment, you better choose between FreeBSD and Opensolaris. You can move pools between these two systems, as long as the pool version does not exceed that which is supported on both. For example you cannot export an OpenSolaris ZFS v14 pool to FreeBSD 8.0 which has pool version 13. But you can export that array to FreeBSD 8.1 which has ZFS version 14.

ZFS is as good as it gets regarding software RAID; it manages to really use the extra performance potential, but also loses performance because of the many safety features like checksums and authenticated metadata. All this is to protect the filesystem from corruption; something other filesystems lack.

The regular Software RAID options under both Linux and FreeBSD do very well too; are extremely fast when properly setup, but imply a 'traditional' filesystem such as Ext3/4, XFS, JFS or UFS in the case of FreeBSD. ZFS is just a totally different class of filesystem in that comparison; not only because it does the RAID part itself.

My web-interface is nearing a first release; it is in testing phase at the moment and most core features of managing ZFS pools (expansion and other stuff) already works. But the install procedure is long and much less convenient than FreeNAS for example.

Still, you could try by setting up a Virtualbox VM and install FreeBSD to it according to my online how-to guides:
http://submesa.com/data/bsd

Then you can set up the webserver and my scripts, and do all ZFS configuring via the web-interface instead. You may still need shell access, for other things than sharing files like a mailserver, but it might simplify your maintenance. Creating snapshots for example.

To read why ZFS is so great, this thread also has a lot of information:
http://hardforum.com/showthread.php?t=1500505

And wikipedia, of course:
http://en.wikipedia.org/wiki/ZFS

 

[Pyroflea]

Awesome, I've got some reading to do Thank you very much.

 

[skittle]

An alternative to ZFS for linux (although not quite as mature) is BTRFS which implements (differently) many of the same features as ZFS (and some of its own 😉 )