DF_

May 6, 2021
4
0
10
Hello,

I have a question concerning taking HDDs on the go, and consolidating files.

Outline

I work in VFX and CGI. I have a studio with two very well-specd Ryzen builds that I use on a daily basis, and at the moment one of those PCs has various HDDs within it that I use for projects; these HDDs are shared on my little LAN, and the other machine reads and writes to them. This means that I can run processes and render on either machine and read/write from/to one central location. These HDDs number 3 in total - one is my WORK drive, for project files, one is my RESOURCES drive, for assets that are linked into 3D scenes, and one is my LIBRARY drive, containing assets that are used in various projects, but not linked directly. This is a slightly slower drive. All are old fashioned HDDs, not SSDs.

I also have another large size portable HDD that I have connected to my main PC at all times, and use Macrium Reflect to run automatic back ups of all 3 drives daily. I have a second large size portable HDD, also running Macrium Reflect, and doing the same backup - only this one I take home with me every night. The idea is to have a backup off-site.

If I have to travel away from my studio and work, I have been taking one of the two backup HDDs with me. I can plug this into my laptop, and use Macrium Reflect to mount one of its backup images - this then displays a virtualised version of the drives I have been backing up; WORK, RESOURCES and LIBRARY, complete with their original drive letters, as if they were actually connected to the laptop. This is brilliant, and lets me continue working with the same files and folders, linking from the 'same' locations when on the move.

So far so good.

The issue is that you cannot write to these Macrium images, just read. So when on the move, I can read from the virtualised drives, but have to save locally to my laptop. When I return to my studio, consolidating and syncing the files that I have edited and changed between the studio PCs and my laptop is a cumbersome process.

Question

I would like a process whereby the two PCs in my studio read and write to a few central drives (at the moment I need 3 seperate ones - as mentioned above). These drives are constantly being backed-up automatically to another set of drives in two ways - 1, purely for a failsafe, and 2, to create a second working copy of each drive. Then, when I am on the move with my laptop, I can take the second working copy of the drives with me, and read and write from/to them with a laptop.

It is important that the PCs in my studio still have access to their drives, in case I need to remote access into them. (I cannot, for example, take their drives with me leaving them with nothing).

When I return to the studio, I want to be able to connect the portable/laptop versions of my drives to the original drives, and run some automated process which will check for new or updated files and folders, and synchronise both so they contain the same.

Does anyone know of a way of achieving this? What software would I need for the syncing, and what HDDs or SSDs would be a good bet for the portable versions? Do I need to build a mini server type thing that is distinct from my two PCs?

Any help and advice is hugely received!

Thank you,
Douglas
 

USAFRet

Titan
Moderator
Cloud, in the middle of the two systems.

Work on A.
When done, upload to the cloud.
Download on B, work.
Repeat in the other direction.


Or, just the drive, without Macrium.
Use an external as your working drive.
 

DF_

May 6, 2021
4
0
10
The issue is that I am working with terabytes of stuff here. Sorry, I neglected to mention that.

Across the three drives, I have over 20 TB of data that I need to read/write from/to. The cloud is just too slow to use for this - to upload and download like that.
 

USAFRet

Titan
Moderator
The issue is that I am working with terabytes of stuff here. Sorry, I neglected to mention that.

Across the three drives, I have over 20 TB of data that I need to read/write from/to. The cloud is just too slow to use for this - to upload and download like that.
Well...Macrium isn't the tool for this.
Use it for what it is good for...Images/backups/cloning.

So, a couple of external drives of sufficient size.
Either work on them directly, or copy the files from each PC when ready.

(and of course Macrium backups as needed)
 

DF_

May 6, 2021
4
0
10
So, a couple of external drives of sufficient size.
Either work on them directly, or copy the files from each PC when ready.

Do you know of any software that can sync changes between two drives? If I copy from drive A to drive B, make a lot of changes, create new files, modify etc on drive B, then want to ensure that drive A is up to date with what is now on drive B ... other than keeping a list and copying the changes back over (which seems a bit archaic), is there a way to streamline this?
 

USAFRet

Titan
Moderator
Do you know of any software that can sync changes between two drives? If I copy from drive A to drive B, make a lot of changes, create new files, modify etc on drive B, then want to ensure that drive A is up to date with what is now on drive B ... other than keeping a list and copying the changes back over (which seems a bit archaic), is there a way to streamline this?
FreeFileSync and SyncBackFree.
 

DF_

May 6, 2021
4
0
10
Thank you for this - much appreciated. This looks like it could be a good solution.