24 Drive JBOD help. SFF-8087 connection help.

[Shardukar]

I am trying to make a 24 drive JBOD (He doesn't want a RAID, I first recommended a Lacie Big 5 NAS with 20TB of storage. I Installed it for him and told him that is enough space to last a long time. Six months later he called me and told me it was full, what should he do now. He digitizes video for a living. ) for a friend. I have a case, and the drives, and the power. The problem that I am having is how to connect all the drive physically to the motherboard. At first I thought that I would use SATA expanders, but that idea didn't work so well. So I have been looking at raid cards with SAS to SATA breakout cables.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816118182

is an example at what I am looking at.

With the 2 x SFF-8087 mini-SAS according to the LSI website "Connect up to 256 SAS and SATA devices with 8 internal 6Gb/s SAS ports" this is the point where I get lost. To me one SFF-8087 port can handle 4 SATA drive so 4 + 4 = 8 SATA devices not 256.

Can someone explain to me what how you can control 256 devices with only 8 physical connections.

Thanks for you help

 

[USAFRet]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

 

[popatim]

controller connects to the expander, Expander connects to the drives
Personally I would grab a used Dell perc6i or IBM M1015

Please make your friend aware that important files still need to be backed up.

 

[TyrOd]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

Digitizing refers to converting tape/vhs to digital media.

Obviously you can easily use 20TB if you store a copy of everything you convert.

That is a really bad business practice. This guy needs a proper enterprise BDR/Datacenter solution.
Trying to set up such a huge system on site for him without proper redundancy and backup is a terrible idea.

Eventually no amount of DIY storage is going to be enough and he'll need archival storage as well.
No offense but hiring you to set up this much storage for him was a bad idea.

In your position I would look into the enterprise BDR market and resell him the most appropriate service from Dell, IBM, etc...

 

[USAFRet]

Digitizing refers to converting tape/vhs to digital media.

Obviously you can easily use 20TB if you store a copy of everything you convert.

Right.

That is a really bad business practice. This guy needs a proper enterprise BDR/Datacenter solution.
Trying to set up such a huge system on site for him without proper redundancy and backup is a terrible idea.

Which is why I said "hmmm..." What exactly are we digitizing here?

20TB in 6 months is far beyond 'winging it', as seems to be the case here.

 

[Shardukar]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

First he use the BIG 5, but is was filled in about six months. It also crashed a lot, he lost the RAID volume twice, that is why he doesn't want a raid. Then I found a used machine, that someone was using as a home theater device, massive case that can support 24 drives. The guy I got the case from only have eight drives hooked up.

http://filmdigitalisering.no/

If you can read Norwegain (or you use Chrome) that is his website.

 

[USAFRet]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

First he use the BIG 5, but is was filled in about six months. It also crashed a lot, he lost the RAID volume twice, that is why he doesn't want a raid. Then I found a used machine, that someone was using as a home theater device, massive case that can support 24 drives. The guy I got the case from only have eight drives hooked up.

http://filmdigitalisering.no/

If you can read Norwegain (or you use Chrome) that is his website.

If this is for an income generating business, then he needs a proper solution. Not someone's ex home movie box.

 

[Shardukar]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

Digitizing refers to converting tape/vhs to digital media.

Obviously you can easily use 20TB if you store a copy of everything you convert.

That is a really bad business practice. This guy needs a proper enterprise BDR/Datacenter solution.
Trying to set up such a huge system on site for him without proper redundancy and backup is a terrible idea.

Eventually no amount of DIY storage is going to be enough and he'll need archival storage as well.
No offense but hiring you to set up this much storage for him was a bad idea.

In your position I would look into the enterprise BDR market and resell him the most appropriate service from Dell, IBM, etc...

Small business, small money. Look into the cost of Enterprise BDR, and compare that cost to a 4TB hard drive. All the data doesn't have to be live all the time. When he fills up a drive it is taken out and put in a filing cabinet. With 24 drive x 4TB that is a year or two worth of live storage. It is allot cheaper then the enterprise approach of endless money. He stores all of his work as a courtesy to his customers, he doesn't use it as a selling point. So when a customer loses the drive that the movies were stored on, my friend has a copy. Back in the day, before hard drive and network storage, we stored all the data on floppy disk that were kept in a desk, the same idea, but now the floppies are 4TB hard disk.

 

[Shardukar]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

First he use the BIG 5, but is was filled in about six months. It also crashed a lot, he lost the RAID volume twice, that is why he doesn't want a raid. Then I found a used machine, that someone was using as a home theater device, massive case that can support 24 drives. The guy I got the case from only have eight drives hooked up.

http://filmdigitalisering.no/

If you can read Norwegain (or you use Chrome) that is his website.

If this is for an income generating business, then he needs a proper solution. Not someone's ex home movie box.

I guess that you have never started your own business, if you had you would know that you don't get to go run out and buy all the top dollar toys you want. Sometimes, you have to get things do on a budget. I asked a question about how to connect 24 drives to a motherboard, and you have not even tired to answer that, just made snide comments. I am disappointed that that a USAF retiree would be more focused on belittling someone then trying to help someone find the solution, and you are a moderator to boot!

 

[Shardukar]

controller connects to the expander, Expander connects to the drives
Personally I would grab a used Dell perc6i or IBM M1015

Please make your friend aware that important files still need to be backed up.

thank you for your help, what expander would you recommend?

 

[USAFRet]

No, I get starting out on a shoestring. And I am NOT promoting some expensive pile of stuff.

However....sometimes there is a lower limit of what will work, and work well.

 

[USAFRet]

Is 20TB every 6 months really needed?
Is 24 discreet drives really needed?
Does the workflow need some adjusting to not need that much space online?

Maybe he does need that much. We do not know.
We do know that a backup of some or all of it will be required. Potentially doubling the space requirement.

Not trying to be snide. But I've run into many, many situations where a customer says "I need X", when they really need something else.

 

[choucove]

You can setup and run multiple hard drives from a single RAID controller utilizing a SAS expander. However, while this sounds like a cheap and simple method to accomplish what you are wanting to do at first, it actually can be quite complex to ensure it is working properly. Certain SAS expanders may have compatibility requirements with specific RAID controllers, certain enclosures, and even certain hard drives. I've seen SAS expanders that only allows the use of SAS hard drives which are much greater cost than standard SATA hard drives meaning it wouldn't be a cost-effective solution for you.

Honestly, if you are not needing huge throughput but lots of expandability, I would look into a rackmount server system to use as your "head" node and connect that to an already existing JBOD unit. There are a LOT of options out there for this. Basically, your primary server is going to be what actually runs everything, it is the server on the network that you can connect to as a file share, iSCSI, or however else you are doing it. This server also has one or more RAID controllers or HBA with external SAS connectors which you can connect to separate JBOD drive shelves. This gives you the capabilities of connecting multiple drives to a single SAS lane. Again, this doesn't give you the greatest performance, but it is much more expandable.

 

[TyrOd]

He wanted 24 drives?
And then 6 months later a 20TB NAS is full?

Before we get to the 'how', what is he doing with this? 'digitizes video' ? hmmm.....

Digitizing refers to converting tape/vhs to digital media.

Obviously you can easily use 20TB if you store a copy of everything you convert.

That is a really bad business practice. This guy needs a proper enterprise BDR/Datacenter solution.
Trying to set up such a huge system on site for him without proper redundancy and backup is a terrible idea.

Eventually no amount of DIY storage is going to be enough and he'll need archival storage as well.
No offense but hiring you to set up this much storage for him was a bad idea.

In your position I would look into the enterprise BDR market and resell him the most appropriate service from Dell, IBM, etc...

Small business, small money. Look into the cost of Enterprise BDR, and compare that cost to a 4TB hard drive. All the data doesn't have to be live all the time. When he fills up a drive it is taken out and put in a filing cabinet. With 24 drive x 4TB that is a year or two worth of live storage. It is allot cheaper then the enterprise approach of endless money. He stores all of his work as a courtesy to his customers, he doesn't use it as a selling point. So when a customer loses the drive that the movies were stored on, my friend has a copy. Back in the day, before hard drive and network storage, we stored all the data on floppy disk that were kept in a desk, the same idea, but now the floppies are 4TB hard disk.

No offense again but your friend doesn't know what he is doing. He is running a backup service for his clients as a courtesy and you want to put it on a 24 X 4 TB non redundant system with NO backup and NO fault tolerance?

This will cost you upwards of $4K USD just for the drives themselves.
Needless to say a proper BDR solution with redundancy and proper backup will cost more for that much capacity BUT that's because nobody in their right mind just keeps 100TB of storage as a "courtesy".

I mean...what do you think is the actual value of doing this for his clients? Do you really believe that a 4TB SATA drive sitting on a shelf is a reliable backup for storing many clients data on? You might as well take that $200 you spend on the drive, split it between your clients so they can spend it towards a proper backup of their own.

Your friend think that he wants to run a DIY datacenter for his clients as a courtesy and it's incredibly naïve.

either invest in a proper BDR/SAN solution or tell your friend that there's no system you can set up for him that will be reliable for only the cost of the storage itself.
Anything else would be irresponsible.

Edit: Choucove's comment is a good start for how to go about a proper iSCSI SAN infrastructure solution if you don't want a third Party BDR solution.

 

[FireWire2]

I am trying to make a 24 drive JBOD (He doesn't want a RAID, I first recommended a Lacie Big 5 NAS with 20TB of storage. I Installed it for him and told him that is enough space to last a long time. Six months later he called me and told me it was full, what should he do now. He digitizes video for a living. ) for a friend. I have a case, and the drives, and the power. The problem that I am having is how to connect all the drive physically to the motherboard. At first I thought that I would use SATA expanders, but that idea didn't work so well. So I have been looking at raid cards with SAS to SATA breakout cables.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816118182

is an example at what I am looking at.

With the 2 x SFF-8087 mini-SAS according to the LSI website "Connect up to 256 SAS and SATA devices with 8 internal 6Gb/s SAS ports" this is the point where I get lost. To me one SFF-8087 port can handle 4 SATA drive so 4 + 4 = 8 SATA devices not 256.

Can someone explain to me what how you can control 256 devices with only 8 physical connections.

Thanks for you help

I hope you are refer JBoD as multiple individual HDD, not as SPAN/BIG volume which combine 24 drives...

Some sleazy manufactures could not do the JBOD with multiple drives, so they call SPAN/BIG as JBoD and confuse lot users, including some IT personnel.

That's being said

To connect 265 drives it's very easy in SAS bus. Just daisy chain those SAS expanders together
So 256x drive you need eleven 24 ports SAS expander like this:
http://www.amazon.com/CHENBRO-CK23601-36-Port-Expander-Controller/dp/B008LA8AO4
and each of them has 2x mini SAS (SFF8087) connectors

Quite a few user has use JBOD for ZFS

 

[popatim]

controller connects to the expander, Expander connects to the drives
Personally I would grab a used Dell perc6i or IBM M1015

Please make your friend aware that important files still need to be backed up.

thank you for your help, what expander would you recommend?

both are good units and both are really LSI made. The perc is more capable but you only need jbod so get whichever is available in your area and/or cheaper. Thye perc is usually $140-ish here and the m1015 is about $110 for newer used stock. I paid $109 for my last m1015 which had a data stamp on it of Aug2013. <sweet>

 

[popatim]

Your friend will need a drive dock to access those drives in the file cabinet. Sitting drives tend to seize up if you leave them too long so pull them out and get them running once every couple of weeks. For long term storage he should start investigating tape media.