[SOLVED] Storage Solution Needed for Editing Large amount of Footage (RAW)

Toggle sidebar Toggle sidebar
D

DragonFlange

Sep 3, 2019
9
0
10
I have been asked by a client to edit 26TB worth of footage into a feature film (not uncommon size). The footage is RAW files, which is why it's so large.

This presents a storage AND editing problem:

I need to store 26TB worth of footage, but ALSO access it. I don't need to play it back necessarily, as I will generate much smaller proxy files to edit with, but the software will still need access to the files even if it's not reading it all the time. And the colour grading part of the job will require using the raw files.

So I am thinking about possible solutions.

I have a DELL Precision PC (dual hexacore, 12 core, CPU, 96GB RAM, Titan X & GTX 970 GPUs, internal drives: 1TB HDD (files/projects), 256SSD (OS), 128SSD (cache drive). I also have a Qnap NAS 12TB RAID 5 (8 useable), but this is maxed out already, as well as everything else really.

So....What should I use for storage and editing?

I was considering the external drive route:
G-Technology 20TB G-RAID with Thunderbolt 3, USB-C (USB 3.1 Gen 2), and HDMI, Removable Dual Drive Storage System.
as it will be large enough (just) to hold the footage, and give me fast enough read speeds to edit the RAW 4k footage (192MBps read) due to the RAID and thunderbolt interface.
https://www.amazon.com/G-Technology...=B072HS2CJN&psc=1&refRID=96PG6P3ZKFKWESXSZM7P

Or I could daisy chain a few of the single external drive G-DRIVE USB-Cs together?
https://shop.westerndigital.com/pro...-technology-g-drive-usb-c-usb-3-1-hdd#0G05678

Or put an internal WD Black 6TB sata inside the PC? But thaty's not really enough space.
https://www.amazon.com/Black-4TB-Pe...&creative=165953&creativeASIN=B0792G331G&th=1

And whether I can actually run these ext drives with a Windows PC (running 7, which I could upgrade to 10 soon I guess).

Would I need this adapter? Plugged into a pci slot? Would that work?
https://www.amazon.com/GIGABYTE-Alpine-Thunderbolt-Components-GC-Alpine/dp/B0722SV69N

Are there any better and cheaper solutions?

Thanks for your time team.
 
Solution
justin.m.beauvais
You could get a few Seagate IronWolf drives and RAID them together. They sell them in sizes up to 16TB each.
https://www.amazon.com/dp/B07SGGWYC1/ref=twister_B01MG9FYNQ?_encoding=UTF8&th=1

Getting two of the 16TB drives would allow you to keep them internally and give you just short of 32TB of storage that you wouldn't need to worry about external transfer speeds. Or you could go with two of the 14TB version to save a little more money, and have just shy of 28TB.

It isn't really cheaper, but it is going to be faster, have less latency, and you won't have to worry about other interfaces. You just need 2 open SATA ports that are RAID capable and 2 SATA power connections.
Sort by date Sort by votes
USAFRet

USAFRet

Titan
Moderator
Mar 16, 2013
171,324
17,829
184,590
QNAP has a 4 bay external module that directly interfaces with a QNAP NAS.
TR-004
https://www.amazon.com/QNAP-TR-004-Enclosure-Attached-hardware/dp/B07K4RC7X9

kNKoQ0M.png


I have one to go with my TS-453a.
Fill it with whatever drives you desire to get the required additional space.


Additionally, add to the PC's internal drive space.
2TB Intel 660p, for instance. Currently under $200.
 
Upvote 1 Downvote
justin.m.beauvais

justin.m.beauvais

Estimable
Dec 15, 2017
1,527
199
3,840
You could get a few Seagate IronWolf drives and RAID them together. They sell them in sizes up to 16TB each.
https://www.amazon.com/dp/B07SGGWYC1/ref=twister_B01MG9FYNQ?_encoding=UTF8&th=1

Getting two of the 16TB drives would allow you to keep them internally and give you just short of 32TB of storage that you wouldn't need to worry about external transfer speeds. Or you could go with two of the 14TB version to save a little more money, and have just shy of 28TB.

It isn't really cheaper, but it is going to be faster, have less latency, and you won't have to worry about other interfaces. You just need 2 open SATA ports that are RAID capable and 2 SATA power connections.
 
Upvote 1 Downvote
Solution
D

DragonFlange

Sep 3, 2019
9
0
10
SkyNetRising said:
Put 3x internal 10TB drives into your pc and set them in RAID0 config (or 4x 10TB in RAID5). This will give you 27.2TB of usable storage.

It's possible your QNAP can support such setup as well.
Click to expand...

I don't think I have space in my tower/motherboard for all that. Have space for 1 HDD & 1 SSD (not counting my OS SSD)
 
Upvote 0 Downvote
D

DragonFlange

Sep 3, 2019
9
0
10
USAFRet said:
QNAP has a 4 bay external module that directly interfaces with a QNAP NAS.
TR-004
https://www.amazon.com/QNAP-TR-004-Enclosure-Attached-hardware/dp/B07K4RC7X9

kNKoQ0M.png


I have one to go with my TS-453a.
Fill it with whatever drives you desire to get the required additional space.


Additionally, add to the PC's internal drive space.
2TB Intel 660p, for instance. Currently under $200.
Click to expand...
I defo want one of these in the future, as I also have the 453a, but it seems too pricey a solution at the mo. Plus, my read/write acces speed to my NAS isn't very fast as it is based on the speed of my ethernet cable coming from my router (bog standard Movistar free router). It's fine for playing movies, but I doubt it will handle heavy editing.
Do you have a way to speed up access to and from NAS?
 
Upvote 0 Downvote
D

DragonFlange

Sep 3, 2019
9
0
10
justin.m.beauvais said:
You could get a few Seagate IronWolf drives and RAID them together. They sell them in sizes up to 16TB each.
https://www.amazon.com/dp/B07SGGWYC1/ref=twister_B01MG9FYNQ?_encoding=UTF8&th=1

Getting two of the 16TB drives would allow you to keep them internally and give you just short of 32TB of storage that you wouldn't need to worry about external transfer speeds. Or you could go with two of the 14TB version to save a little more money, and have just shy of 28TB.

It isn't really cheaper, but it is going to be faster, have less latency, and you won't have to worry about other interfaces. You just need 2 open SATA ports that are RAID capable and 2 SATA power connections.
Click to expand...
Are these fast enough? If they are optimised for NAS, are they ok to put inside my PC? Isn't the WD 6TB Black better for editing?

If these drives are just as good, the getting two of them would be great, although it may better to keep them seperate, unRAIDED, because it's typically better to have RAW files, scratch disks, project files & movie files on one drive, and caches on another. Or so I've read...Can you shed any light on this?
 
Upvote 0 Downvote
D

DragonFlange

Sep 3, 2019
9
0
10
Thanks for the replies so far everyone.

No-one seems tempted by the G-Tech external drive solution. How come?

If I DID want to go external (less faff) could I get a Thunderbolt adapter and use my pci port on PC? And use an external Thunderbolt drive like the G-Tech?
 
Last edited:
Upvote 0 Downvote
USAFRet

USAFRet

Titan
Moderator
Mar 16, 2013
171,324
17,829
184,590
Being realistic, how much of this 26TB needs to be online and in edit mode at any one time?
All, some, a little?

And if the PC only has space for 2 more internal drives, you might need to up that.
Internal storage trumps any external thing.
 
Upvote 1 Downvote
D

DragonFlange

Sep 3, 2019
9
0
10
USAFRet said:
The 4 bay QNAP enclosure I linked above is only $200, fill it with whatever drives you want, works natively with your current QNAP NAS, and will do the RAID 0, 1, 5, whatever.
Click to expand...

But my access to my NAS files is not very fast, I'm guessing because it is bottlenecked by the router I have? How do I speed this up? I can't access the NAS files straight from the bay can I?
 
Upvote 0 Downvote
You must log in or register to reply here.

TRENDING THREADS

Latest posts

Moderators online

Share this page

COMPANY
RESOURCES
FOLLOW
Top Bottom