[SOLVED] What storage solution or a data intense hobby project?

[Chris8080]

Hello,

for a hobby project (limited budget that is ), I'm in need for a storage solution.
The project is about archiving and analyzing websites.
Every month, I'm generating around 100GB of zipped files which I need to analyze two times soon after downloading and again later on, maybe.

At the moment, I use my Laptop for that and then, manually move the archived files to two external 2.5 HDD that I got out of old laptops (2 times for having a backup).

I'm fine with a slow transfer speed
The storage doesn't have to be online all the time
I would like to be able to scale easily

I thought about buying 4 - 8 500GB 2nd hand drives and plugging them into an existing tower and installing FreeNAS and switch the tower on on demand - mostly for learning sth about FreeNAS and using a pool of disks.
Alternatively, the price per GB for 2nd hand 4TB is only a bit higher and I'd need only two disks which would make things probably easier.

What would be your suggestion for a non critical / hobby / learn project?
What would be your suggestion, if that project would scale up and even generate some money later on?

 

[RealBeast]

FreeNAS is an excellent choice if you have the time and patience, although I would buy new drives. If you take your time and shop around you can usually find some pretty good deals. I would start with at least 3 drive so that you can use at least single parity (z1), which is similar to RAID 5 in conventional adapter based arrays.

If you don't have the time or patience for FreeNAS look at something like an Adaptec 6805T/. I buy most of my adapters out of Hong Kong brand new for very cheap prices and they are the real thing, like THIS new one for around $30. You will then need one or two fanout cables (SFF-8087 to 4 SATA or SAS connectors depending on the drives you buy, most likely SATA). Each cable can handle 4 drives so you could start with one. I buy those out of China too for about $20 each, you just need to be sure to buy the correct one. Then just add 3 or 4 preferably identical drives and with some reading build the array. Any old parts with a free PCIe x8 or x16 running at x8 usually using internal graphics and a little ram and a small old SSD for the OS is sufficient. I use real old stuff for that usually.

And then there is the easiest solution -- buy a prebuilt NAS box, but there is not much learning experience there and it will cost more. Still quite viable.

While I do some of this for real money, I do it for a hobby at home and my latest build a week ago that is still loading data was an Adaptec 8805 with 8 12TB SAS WD enterprise drives. The total cost was $2300 and a few spare old parts for 66TB of dual parity storage. I spent a few months shopping to get everything as cheap as possible. My 4th home array as I am a real backup kind of guy with cold storage of everything. So be careful -- it can become an expensive hobby.

 

[Chris8080]

FreeNAS is an excellent choice ... So be careful -- it can become an expensive hobby.

Thank you very much for the detailed explanation, that got me further in my research and I guess I'll give FreeNAS a try.

 

[RealBeast]

FreeNAS is excellent, good choice.