Server & storage solutions for small design studio

[gitgit42]

Hi folks. I'm just an artist with very limited knowledge of servers and networking stuffs. So I'm wondering if anyone would be kind enough to recommend possible network/server /storage set ups for a small studio of about 4 -to- 12 artists.? We'll mostly be doing computer graphics(3d/broadcast) and rendering with occasional video editing utilizing off the shelf tools from Autodesk and Adobe. We don't want to go above 10 grand for this server/storage budget starting out from scratch(4 artists with maybe 10 terabyte of working space) if this is even doable(?). Any advice would be very much appreciated. Apologies in advance if I could not reply in timely manner since I don't frequent this community often. Thanks again.
Cheers!
Ron

 

[asoroka]

You need to detail your requirements.

So 10 TB of storage is no problem.

What kind of network. wireless or wired? You will need to get professional cabling which is not cheap, your insurance may not cover you if someone trips over a loose cable. So if you can, use wireless unless your office is already wired.

I assume that you will be doing your computational work on individual workstations, or did you envision a powerful shared application server?

So for network sizing you will need to know how much data is going to flow between each node in your network.

At the very least you will need a good wireless router $300AUD
a storage device (Synology DS418 DiskStation 4-Bay NAS + 2 10TB disks mirrored) $1500AUD

Printer?

What about data backup (a few USB drives?)

What will you run your software on?

I recommend separating storage from any computational servers

 

[gitgit42]

Thanks for replying to this.

You are right, we do intend to keep computation on the individual machines
- coffeelake/ryzen/nvidia.

So this server (if that's even the correct term) is for fast reading and writing of images mostly. Or should it be called storage?

Unfortunately I have no idea what the data traffic is going to be needing for this.. But would you recommend this synology model to be able to serve 4 simultaneous access for both fast read and write? Any other model we should consider to be future proof for 12 simultaneous read/write access?

Thanks again.

 

[kanewolf]

Synology specializes in network attached storage (NAS). This is a device that is managed via a webserver and provides shared storage on your network. There are other vendors with similar products, QNAP and Thecus are the main competitors. Multiple users would benefit from a 10 gigabit network connection to the storage. Are your workstations connected by a wired network today?

 

[gitgit42]

Synology specializes in network attached storage (NAS). This is a device that is managed via a webserver and provides shared storage on your network. There are other vendors with similar products, QNAP and Thecus are the main competitors. Multiple users would benefit from a 10 gigabit network connection to the storage. Are your workstations connected by a wired network today?

Thanks for explaining this kanewolf. It's great to know that there are ready solutions for small players today. Would you recommend Synology over the Qnap or Thecus?

As for how I'm intending to set up. I hope to have a closed wired system for about 12 workstations with direct access to fast file server/storage. No direct access to internet from any of the machines hooked up to the network for security reason.

But hope to have an air gap with internet in future.

Any specific NAS machine would you recommend?

Thanks again for your time kanewolf.

 

[failboat]

In addition to 10G network/NAS you might want to look into offsite tape vaulting.
At a certain point it's going to be cheaper than trying to cloud backup.
If very high speed uploads are available to you then cloud might work.

You probably want two NAS onsite at least. One can be specced for fast current project storage + backup and the second for backup.
raid 50 or 60. 4-7 disks per raid 5. The more disks in each raid will lower your costs, but when one fails the more disks you have the more chances of failure. random r/w speeds diminish with more disks. raid 10 is better, but it will cost a lot more per TB, it's 50% usable space. Anytime a disk goes down there is already a copy, if both disks in the raid 1 go down all is lost in that pool. In raid 5,6 if a disk goes down performance drops a lot and if any disk fails it's all gone. New 10TB disks take days to resilver. If one NAS breaks and the other drops a disk it can get scary fast. Maybe even a 3rd NAS for short term backup if all your projects are short duration, but high losses if you lose data. As stuff goes to tape you can move it off the 3rd NAS.

You can come up with tiers of how valuable project data is and then come up with different storage/backup solutions per tier. If something like wedding photos were lost before they were sent to the client it would be really bad.

If people are going to be working off the files for a week or two then a short term high speed pool would be nice.
Stuff should copy to both backups asap as it moves over.

 

[gitgit42]

In addition to 10G network/NAS you might want to look into offsite tape vaulting.
At a certain point it's going to be cheaper than trying to cloud backup.
If very high speed uploads are available to you then cloud might work.

You probably want two NAS onsite at least. One can be specced for fast current project storage + backup and the second for backup.
raid 50 or 60. 4-7 disks per raid 5. The more disks in each raid will lower your costs, but when one fails the more disks you have the more chances of failure. random r/w speeds diminish with more disks. raid 10 is better, but it will cost a lot more per TB, it's 50% usable space. Anytime a disk goes down there is already a copy, if both disks in the raid 1 go down all is lost in that pool. In raid 5,6 if a disk goes down performance drops a lot and if any disk fails it's all gone. New 10TB disks take days to resilver. If one NAS breaks and the other drops a disk it can get scary fast.

If people are going to be working off the files for a week or two then a short term high speed pool would be nice.
Stuff should copy to both backups asap as it moves over.

That's very good to know and thanks for the advice! I really appreciate it. How can I set up this short term high speed pool? Within an individual NAS using ssd? Or entirely different device running parallel to the main NAS.? sorry I don't understand these network stuff too well. Thanks again for your time. Cheers!

 

[failboat]

In addition to 10G network/NAS you might want to look into offsite tape vaulting.
At a certain point it's going to be cheaper than trying to cloud backup.
If very high speed uploads are available to you then cloud might work.

You probably want two NAS onsite at least. One can be specced for fast current project storage + backup and the second for backup.
raid 50 or 60. 4-7 disks per raid 5. The more disks in each raid will lower your costs, but when one fails the more disks you have the more chances of failure. random r/w speeds diminish with more disks. raid 10 is better, but it will cost a lot more per TB, it's 50% usable space. Anytime a disk goes down there is already a copy, if both disks in the raid 1 go down all is lost in that pool. In raid 5,6 if a disk goes down performance drops a lot and if any disk fails it's all gone. New 10TB disks take days to resilver. If one NAS breaks and the other drops a disk it can get scary fast.

If people are going to be working off the files for a week or two then a short term high speed pool would be nice.
Stuff should copy to both backups asap as it moves over.

That's very good to know and thanks for the advice! I really appreciate it. How can I set up this short term high speed pool? Within an individual NAS using ssd? Or entirely different device running parallel to the main NAS.? sorry I don't understand these network stuff too well. Thanks again for your time. Cheers!

You can have multiple storage pools per NAS. a few SSD in raid would be very fast.

Before doing that you need to figure out what time savings it will actually provide.
I'm not sure how those programs interact with the NAS.
If the working file is saved on the NAS or moved to it.
How big the files are. How many files.

If the file is just storage locally on each workstation and backup in real time then seq speeds are all that really matters.
Spinning disks hit great seq speeds in raid. SSD have great random speeds and seq speeds. seq speeds will be capped by the network 1G or 10G.

 

[gitgit42]

Thanks again for the input failboat, I appreciate it a lot. I will go back to crunching numbers. Cheers!

 

[failboat]

Thanks again for the input failboat, I appreciate it a lot. I will go back to crunching numbers. Cheers!

If they are working off of storage on their workstations and just need backups. You can use slow disks and 1G network.
You would only want 10G network if they are moving files to storage where it's being worked from. For backups people probably aren't waiting for them or anything. So if it's slow it won't matter.

10G and fast network storage will cost you a lot of money.
It could be cheaper than putting huge SSD in each persons workstations though.

 

[kanewolf]

I have personally used Synology. Other moderators here have QNAP. QNAP usually has features at lower price points. Both vendors are very good about features and updates. The one thing to keep in mind if you are looking at small business NAS units is that they are sold without disks. The price you see is the chassis price. Disks are separate. That is good because you are free to pick any disks you want.

If you just are looking at Synology's webpage you want to start with the "plus" line of products. Then depending on your physical environment, you might look at a DS1817+ or the rack mount equivalent.

You do need to think long and hard about backups and physical security of this storage. Don't think that your data is safe because it is on RAID. Here is an article that I like talking about that -- https://www.smallnetbuilder.com/nas/nas-features/31745-data-recovery-tales-raid-is-not-backup
So backups are a must. But physical security of the NAS and a good UPS for the NAS should also be figured into your budget.

 

[gitgit42]

Thanks again for the input failboat, I appreciate it a lot. I will go back to crunching numbers. Cheers!

If they are working off of storage on their workstations and just need backups. You can use slow disks and 1G network.
You would only want 10G network if they are moving files to storage where it's being worked from. For backups people probably aren't waiting for them or anything. So if it's slow it won't matter.

10G and fast network storage will cost you a lot of money.
It could be cheaper than putting huge SSD in each persons workstations though.

Thanks again and do you have any parts or specific devices to recommend for fast network storage for working off the servers? Image files will be read/write off the storage/server simultaneously by about 10 artists.

 

[failboat]

Thanks again for the input failboat, I appreciate it a lot. I will go back to crunching numbers. Cheers!

If they are working off of storage on their workstations and just need backups. You can use slow disks and 1G network.
You would only want 10G network if they are moving files to storage where it's being worked from. For backups people probably aren't waiting for them or anything. So if it's slow it won't matter.

10G and fast network storage will cost you a lot of money.
It could be cheaper than putting huge SSD in each persons workstations though.

Thanks again and do you have any parts or specific devices to recommend for fast network storage for working off the servers? Image files will be read/write off the storage/server simultaneously by about 10 artists.

Hitting the speeds for a production storage environment is a lot more difficult to predict. You really need to find someone who is using the same setup and ask them for advice. Sizing the ram, storage, lc2arc cache, and slog or no slog. Putting in the parts is one thing, but being your own sysadmin is another. If a drive goes down you need a good system in place to identify it and resolve it.

Do you have an examples of what your doing when you have delays related to storage or network? If you have an old pc you can build your own test server. Drop a SSD in it for storage and use what RAM you have and load SAMBA. Then you at least have a reference. With RAM consumer platforms are limited. So if it turns out you need > 64GB then you have to go with Xeon E5. Ram is crazy expensive, but not enough can cripple you and too much is just a waste.