[SOLVED] Best way to run RAID

[neverknowu]

Hi there,
I need to get larger, fast storage on my PC. Right now I have several year old 2 Samsung pro 512gb SSDs in RAID0 using Windows. I'm thinking of getting 3 more of the same drives and running them all on RAID0. I've never created a RAID with so many drives before so I wonder if that's the best thing for me to do.
Can I do it on Windows Asus x99-e ws usb3.1? Is that the best choice? Or should I be using some sort of a RAID controller?
Is there any reason I shouldn't use SSDs?
I'd like to keep it all on board, as I do travel with my system to work. So a raid like a Promise or something of that nature isn't ideal. Also, I'd like it to be as fast as possible, and I don't feel like ethernet connections will be good enough (could be wrong, still trying to get my mind wrapped around gpbs and drive speeds)
This is for reading and writing/playback of large video files.

 

[Eximo]

Well, Windows is the worst way.

Your motherboard has a raid controller you could have been using this whole time, probably more than one, but certainly the chipset has an Intel RAID controller built in. What you are doing now is software raid. Chipset raid is a little better, still uses the CPU for processing, but all of those tasks are invisible to the OS. Best would be a discrete RAID controller with its own processor and ram, these are rather expensive though.

There is little point in RAID0 for SSDs in the first place, they are essentially already configured that way internally. Only noticeable speed difference is under a benchmark.

If you want more speed, just pick up a PCIe SSD (as long as you don't need boot). Should be able to get a card with that will take multiple M.2 NVMe drives, or a discrete PCIe card like Intel Optane.

1Gbps is approximately 80-100 MB/s in practice. So unless you have a 10GB network at your disposal, such speeds are already pointless.

 

[USAFRet]

2x SSD in RAID 0, and you want to go to 5x?

I can't think of any good reason to do that.
Even at 2 of them.

 

[RealBeast]

First, I strongly suggest that you not use motherboard based RAID 0 ever -- the odds are quite high for failure due to any simple bios hiccup.

RAID 0 is not really recommended by anyone with any experience anymore -- too much chance of failure, and if one disk goes all data is lost -- and yes even SSDs can die.

If you really want to do RAID after you consider all alternative, use a real RAID adapter -- a Adaptec or LSI (don't mess with junk adapters).

And I agree that RAIDing SSDs is not necessary. What is your storage need in size and what kind of use, i.e. how fast does it need to be?

I use an 8 drive RAID 6 array of 8TB drives and it is plenty fast over a simple 10Gbps SFP+ connection, but it is highly unlikely that you would need that sort of set up. A single 10TB drive is a pretty fast storage form in my video work (with the program and scratch disk on separate SSDs), then major projects are backed up to the larger array and an offline storage for secondary backup. It really depends on your needs.

 

[logainofhades]

Raiding SSD's is pointless, unless you are using a bunch of them, as a large/fast scratch disk, for video editing and such. For Windows use, not worth it.

 

[neverknowu]

2x SSD in RAID 0, and you want to go to 5x?

I can't think of any good reason to do that.
Even at 2 of them.

It's for Read/Write of DPX 4k 10bit video files while doing effects and having them play in real time.

 

[neverknowu]

First, I strongly suggest that you not use motherboard based RAID 0 ever -- the odds are quite high for failure due to any simple bios hiccup.

RAID 0 is not really recommended by anyone with any experience anymore -- too much chance of failure, and if one disk goes all data is lost -- and yes even SSDs can die.

If you really want to do RAID after you consider all alternative, use a real RAID adapter -- a Adaptec or LSI (don't mess with junk adapters).

And I agree that RAIDing SSDs is not necessary. What is your storage need in size and what kind of use, i.e. how fast does it need to be?

I use an 8 drive RAID 6 array of 8TB drives and it is plenty fast over a simple 10Gbps SFP+ connection, but it is highly unlikely that you would need that sort of set up. A single 10TB drive is a pretty fast storage form in my video work (with the program and scratch disk on separate SSDs), then major projects are backed up to the larger array and an offline storage for secondary backup. It really depends on your needs.

I'd like to get around a 1200 and up read/write speed for DPX 4k 10bit video files for real time playback while doing vfx on them. Davinci Resolve raw RED camera playback, that kind of thing.

Yeah, I don't know anything about RAID adapters (like which ones are the best to get) or if not RAID0, then what?

It's not for a permanent storage. It's for one job, then I'd blow the media off after I've rendered to an external drive. I'm getting multiple jobs that need 250GB to 2.5 TB per job so I'm trying to come up with a solution for my system and have it portable at the same time.

 

[USAFRet]

It's for Read/Write of DPX 4k 10bit video files while doing effects and having them play in real time.

How are these drives connected? Internally in the PC?

Currently, you have 2x. Why the desire to go to 5x?
Is the 1TB not enough space for 1 project at a time?

 

[RealBeast]

How portable do you need to be and what's your budget?

 

[neverknowu]

How are these drives connected? Internally in the PC?

Currently, you have 2x. Why the desire to go to 5x?
Is the 1TB not enough space for 1 project at a time?

No, unfortunately not. I get a single project in that's anywhere from 150GB to 2 TB, then I'm still working on multiple projects at the same time. It's really a drag.

 

[neverknowu]

How portable do you need to be and what's your budget?

I'd like to figure something out that's $1,200 tops...but I don't know how to weigh out my options. I see NVMe PCI Raid cards but those get to be $2000 and over. If I have to do it though it would be good to know that that's what I have to spend.

Portable, as in, throw my computer in a case in a car or airplane.

 

[logainofhades]

A possible raid card solution, to fit in your current pc?

https://smile.amazon.com/High-Point-SSD7101A-1-Dedicated-Controller/dp/B073W71K4Z

 

[neverknowu]

Also, I'm curious as to why not SSDs? I have raid 0 ssd x2 and I'm getting around 800 read write speeds. That's good, right?

And if not RAID 0, Raid 5?

I'm running out of PCIe lanes too, as I already have 2 GPUs and a couple other small cards. So maybe not NVMe if having each one is x4?

Also took a look at this:
https://www.bhphotovideo.com/c/product/1269780-REG/highpoint_rocketraid_840a_pcie_3_0.html

 

[logainofhades]

Someone that does have a raid array, for scratch disk work. I think you need a HEDT type rig, for what you really are wanting to do, due to PCI-E lane limits.

 

[USAFRet]

Also, I'm curious as to why not SSDs? I have raid 0 ssd x2 and I'm getting around 800 read write speeds. That's good, right?

Read/write speeds in a benchmark are one thing.
User facing performance can be quite different.

 

[RealBeast]

I'd like to figure something out that's $1,200 tops...but I don't know how to weigh out my options. I see NVMe PCI Raid cards but those get to be $2000 and over. If I have to do it though it would be good to know that that's what I have to spend.

Portable, as in, throw my computer in a case in a car or airplane.

Between your budget limitation and portability needs, forget about RAID arrays and PCIe NVME cards.

You should look at one NVME drive (as that is all your MB supports) and your drives should probably be a 2 GB drive most likely for your output (although there is a 3.84GB available at higher cost), a decent sized scratch SSD (like 512GB-1TB) and an SSD for your OS/program, which I would expect that you already have. Which drive is the NVME depends on what you already have and which would benefit the most from a faster speed -- either the output drive or the scratch drive unless you are already using the NVME for your OS drive.

It always helps if you provide a lot of information on what you already have so everyone doesn't have to guess, so do you already have an Asus x99-e ws usb3.1 . Do you need to include any other parts in your $1200 budget?

 

[USAFRet]

As above.
2TB NVMe project drive.
500GB-1TB SATA III OS and application drive.
500GB SATA III scratch space drive.

Solid, stable, much more portable, less hassle, less fail potential.
And of course, whatever backup system and routine you implement.

 

[neverknowu]

Between your budget limitation and portability needs, forget about RAID arrays and PCIe NVME cards.

You should look at one NVME drive (as that is all your MB supports) and your drives should probably be a 2 GB drive most likely for your output (although there is a 3.84GB available at higher cost), a decent sized scratch SSD (like 512GB-1TB) and an SSD for your OS/program, which I would expect that you already have. Which drive is the NVME depends on what you already have and which would benefit the most from a faster speed -- either the output drive or the scratch drive unless you are already using the NVME for your OS drive.

It always helps if you provide a lot of information on what you already have so everyone doesn't have to guess, so do you already have an Asus x99-e ws usb3.1 . Do you need to include any other parts in your $1200 budget?

I'm sorry, you're right. I should have listed my full computer specs. I also didn't predict that I would need to delve into explaining my needs so much...kinda feel like an idiot. I really appreciate everyone's time and knowledge.

My specs:
Asus x99-e ws usb 3.1
Intel Core i7-5960x
64GB RAM DDR4 2133
M2 512GB 950 Samsung Pro
1x Samsung SSD 840 Pro 512gb for system drive
2x Samsung SSD 840 Pro 512gb that are Raid0
2x EVGA 2080ti RTX
Blackmagic Decklink
USB C Expansion Card

I read the video media off of my M2 drive. 512GB is not large enough at present, so I know I need to update that. When I finish my work, I render off to the RAID0 (as I need to render on to a second drive so there's no bottleneck between the drive I'm reading and writing to). I'm writing files that can be enormous. Up to 5.5K 16Bit DPX files, or 30minutes of 4k DPX files, etc.. The faster it can write the files, the faster I can get a copy going to dump the files off of my computer, make the client happy, go home and be with the family.

It would be really nice if there was a second drive that was large and fast to hold the renders. (I'd also like a mobo that has TB 3, but that's another story(yes I tried installing the AsusTB card...it never worked)). So my direct question at the beginning was to figure out options for this render drive, and my first idea was hell, why not get as many SSDs running Raid0. Sounds like that's not a good option though. And also, from the earlier comments about benchmark performance, maybe a 2TB Samsung pro SSD would be better as a render drive than having multiple SSDs running RAID0??? I don't know enough to know this to be true.

I had a job recently that had 2.5TB I had to access immediately, so that is very concerning to me (pro m2 drives don't even come bigger than that!!). What I did was to access the external drive via USBC connection and render proxy files on my M2 drive as I worked. Not ideal, but it worked fine. How though, in the future, do I handle a job like that better? That's a massive amount of data to have to access right away. Maybe if I make the render drive large enough, I can use the render drive to read video files on the short term.

Maybe I'm pushing this computer configuration too much, and need to update too. When I say $1200...I'd LIKE that to be my budget. But if I need to spend more/save, then it would be good to know that to do the BEST thing rather than put bandaids (spending a lower amount of money).

 

[neverknowu]

Someone that does have a raid array, for scratch disk work. I think you need a HEDT type rig, for what you really are wanting to do, due to PCI-E lane limits.

This was a very helpful video to learn more about the nitty-gritty cost vs performance of M2. Thank you.

 

[RealBeast]

Great, now you can consider whether you make enough (like is this a full time or at least lucrative part time job) that it is worth a new build using as many old parts as possible. You would really benefit from a lot more PCIe lanes, which some of the newer AMD Zen based systems have that would allow you to use more NVME storage along with your 2 high end video cards.

Just a thought, but you may want to post a possible new build thread for a workstation that will meet the needs that are getting perhaps too demanding for your current build.

Your other choice would be to spend some money now on more storage, but for NVME you would need adapters, but you may not have enough lanes left.