Question An inconvenience with mirror backups?

[robert600]

I have an extensive collection of movies which I'm constantly adding to. I store them in alphebetical folders. For finding a given movie easily, when a given letter has more than say 30 movies or so, I break the folder into smaller ones. For instance, when the A folder gets to 30 movies, I break it into 2 new folders Aa-Al and Am-Az. This works ok but there's an inconvenience with mirror backups.
On the original Drives, I simply move the appropriate movies into the new folders and then delete the original folder (in the case above ... that would be the original A folder). Moving the files is of course very quick. But, then when I do a mirror backup ... on the backup disk, rather than moving the files, the software deletes the A folder and then copies the new folders and their contents from the original drive ... this can take a rather long time and seems unnecessary. Is there any way to change this copying behaviour into a moving one or is that asking too much?

 

[Grobe]

I have an extensive collection of movies which I'm constantly adding to. I store them in alphebetical folders. For finding a given movie easily, when a given letter has more than say 30 movies or so, I break the folder into smaller ones. For instance, when the A folder gets to 30 movies, I break it into 2 new folders Aa-Al and Am-Az. This works ok but there's an inconvenience with mirror backups.
On the original Drives, I simply move the appropriate movies into the new folders and then delete the original folder (in the case above ... that would be the original A folder). Moving the files is of course very quick. But, then when I do a mirror backup ... on the backup disk, rather than moving the files, the software deletes the A folder and then copies the new folders and their contents from the original drive ... this can take a rather long time and seems unnecessary. Is there any way to change this copying behaviour into a moving one or is that asking too much?

FreeFileSync should be able to detect moved files:

https://freefilesync.org/forum/viewtopic.php?t=8969

 

[Ralston18]

That seems to be logical but is based on a "file cabinet" like system. Physical.

At most you could use different folders to divide movie types but that is likely to become cumbersome.

Go for simplicity. Let the software do the work.

Consider that if all the movies were in one big folder then all that you would need to do to find any given movie is to initiate a wild card (usually an *) search based on some letters in the movie title. And not keep drilling down through the physical equivalent of file cabinet, drawer, folder, sub-folder.....

Removes the necessity to subdivide folders at the 30 movie "mark" (+ or - a few movies as required).

Which often runs afoul of things like titles starting (for example) with "The". Doubt that you put all of those movies into a "T" or "Th" labled folder.

Where "The car" would need to be moved to the "C" or "Ca" folder and "The Mask" would need to be moved to the "M" or "Ma" folder.

= = = =

The search results may result (depending on the search criteria) more than one movie.

However, you simply click the desired movie file and thus view the movie accordingly.

Searching can be speeded up by using indexing. The system will do that as well if enabled to do so.

My thought and suggestion is to reconsider the movie filing system/process.

Little or no inconvenience necessary.

Probably no need at all for any third party software beyond what is necessary for just backing up the drive or folder where the movies are stored.

Just my thoughts on the matter.

 

[robert600]

FreeFileSync should be able to detect moved files:

https://freefilesync.org/forum/viewtopic.php?t=8969

I agree that it should but doesn't that link indicate that it doesn't? Maybe I'm reading it incorrectly?

 

[robert600]

Consider that if all the movies were in one big folder then all that you would need to do to find any given movie is to initiate a wild card (usually an *) search based on some letters in the movie title. And not keep drilling down through the physical equivalent of file cabinet, drawer, folder, sub-folder.....

Interesting thought ... that would work well on a HTPC but .... 95% of the time I'm accessing these movies through Kodi on a shield. I'm not sure Kodi allows for searching local files and even it did, specifying a search without a keyboard would be cumbersome. My way ... 5 or 6 button pushes on the remote and the movie is playing.

 

[hotaru.hino]

There's no way for backup software to mirror the actions you did. That is, if you split your "A" folder to "Aa-Al" and "Am-Az" and moved the video files from there, the software has no way of knowing you did that or how to go about it. It only knows that there's a difference between the source and the destination.

You could create a Powershell script or similar that can create the directories and move the files ahead of time in the destination to prepare, but that has to be updated any time your folder structure at the source changes, which becomes inconvenient.

To me, since backing up is typically a "do and forget" operation, I don't really care how long it takes. What matters to me more is that it's easy to do. So if the backup solution has to do a "copy the source to the destination and delete the differences at the destination", so be it.

 

[Lafong]

then when I do a mirror backup ....................Is there any way to change this copying behaviour into a moving one or is that asking too much?

I'm not sure I'm following you....

You say this is a "backup" procedure.

If you move files from the source drive to the target drive, the 2 drives are no longer mirrored and are not replicas.

How would that be a backup in which you have 2 copies of all the movies.

Free File Sync will certainly mirror the 2 drives. I've used it several times a day for exactly that purpose. The target drive becomes a replica of the source drive every time I run the application. Which is exactly what I want.

I must be misunderstanding something?

 

[Ralston18]

I will need to defer to Kodi users regarding saving, storing, finding, and retrieving movies.

 

[hotaru.hino]

I must be misunderstanding something?

I think what OP wants is the software to perform the actions they did on the source side rather than copy the files over and delete the differences.

So for example if they want to split the contents of the folder with movies that start with the letter "A" into "Aa-Al" and "Am-Az", they want the software to create the folders and move the movie files over, rather than copy the "Aa-Al" and "Am-Az" folders over and delete the "A" folder.

I don't know why they're concerned about this other than time or paranoia about extra wear on the drive.

 

[Grobe]

There's no way for backup software to mirror the actions you did. That is, if you split your "A" folder to "Aa-Al" and "Am-Az" and moved the video files from there, the software has no way of knowing you did that or how to go about it. It only knows that there's a difference between the source and the destination.

What I understand about FreeFileSync is that it have a mode where it utilize a database, - I guess it use an algorithm using check sums, file size and date stamp as a measurement to check if two files with different names and/or in different folder - in fact is the same.

 

[hotaru.hino]

What I understand about FreeFileSync is that it have a mode where it utilize a database, - I guess it use an algorithm using check sums, file size and date stamp as a measurement to check if two files with different names and/or in different folder - in fact is the same.

I did a test to see how it would handle the situation OP is presenting. It'll just delete the old folder at the destination and copy the new folders over, even if the files were only moved at the source location.

Either way, there's no way I'm aware of for backup software to capture file system operations. And even then, allowing software to capture file system operations seems kind of like a security concern to me.

 

[robert600]

I did a test to see how it would handle the situation OP is presenting. It'll just delete the old folder at the destination and copy the new folders over, even if the files were only moved at the source location.

Yes, thank you ... you've understood perfectly what I'm doing. I"m a bit dyslexic so I probably didn't explain it very clearly.

I don't know why they're concerned about this other than time or paranoia about extra wear on the drive.

It's a combination of all these coupled with a desire to do things effeciently. Bear in mind I'm working with video files ... not say ... documents. What with 4K and Atmos soundtracks ... movie files can easily be 35 Gb in size ... so splitting a folder with 30 such files represents a lot of data (you can do the math better than me) and copying rather than moving seems like a brute force approach but if that's the only way back upsoftware can do things, I'll just have to live with it .

The only work around I've come up with is to manually do the exact same splitting of folders and files on both the source and destination ... then run the mirror and if I'v done the splitting correctly on both ... nothing gets transferred.

 

[hotaru.hino]

The only work around I've come up with is to manually do the exact same splitting of folders and files on both the source and destination ... then run the mirror and if I'v done the splitting correctly on both ... nothing gets transferred.

There's a way to automate this, but it requires knowing how to write a PowerShell script... and general programming knowledge.

 

[MWink64]

FreeFileSync should be able to detect moved files:

https://freefilesync.org/forum/viewtopic.php?t=8969

FreeFileSync can update moved/renamed files on filesystems that support file IDs, on the second sync, when "use database file to detect changes" is enabled. Here is a more appropriate link.

What I understand about FreeFileSync is that it have a mode where it utilize a database, - I guess it use an algorithm using check sums, file size and date stamp as a measurement to check if two files with different names and/or in different folder - in fact is the same.

Yes, but it's not using checksums, it's going by file ID.

I did a test to see how it would handle the situation OP is presenting. It'll just delete the old folder at the destination and copy the new folders over, even if the files were only moved at the source location.

Either way, there's no way I'm aware of for backup software to capture file system operations. And even then, allowing software to capture file system operations seems kind of like a security concern to me.

FreeFileSync absolutely will work. I've used it countless times for this kind of situation. I'm guessing your test did not meet all of the requirements noted above. BTW, it's not capturing filesystem operations, it's tracking file IDs.

Yes, thank you ... you've understood perfectly what I'm doing. I"m a bit dyslexic so I probably didn't explain it very clearly.

It's a combination of all these coupled with a desire to do things effeciently. Bear in mind I'm working with video files ... not say ... documents. What with 4K and Atmos soundtracks ... movie files can easily be 35 Gb in size ... so splitting a folder with 30 such files represents a lot of data (you can do the math better than me) and copying rather than moving seems like a brute force approach but if that's the only way back upsoftware can do things, I'll just have to live with it .

The only work around I've come up with is to manually do the exact same splitting of folders and files on both the source and destination ... then run the mirror and if I'v done the splitting correctly on both ... nothing gets transferred.

FreeFileSync will almost certainly do exactly what you want. I'd suggest using "mirror" mode. Don't forget to enable "use database file to detect changes" and perform a synchronization (not just compare) at least once, before you start moving files around.

 

[robert600]

FreeFileSync will almost certainly do exactly what you want. I'd suggest using "mirror" mode. Don't forget to enable "use database file to detect changes" and perform a synchronization (not just compare) at least once, before you start moving files around.

YES!!! I just completed a small test of this and it worked perfectly. Thanks so much! It makes me wonder why "use database file to detect change" isn't enabled by default?

 

[robert600]

FreeFileSync will almost certainly do exactly what you want. I'd suggest using "mirror" mode. Don't forget to enable "use database file to detect changes" and perform a synchronization (not just compare) at least once, before you start moving files around.

They keep changing things here ... I have no idea how to mark the above as "best answer" or "problem solved" or whatever.

 

[USAFRet]

They keep changing things here ... I have no idea how to mark the above as "best answer" or "problem solved" or whatever.

Try it now.
(checkmark at the right)

 

[robert600]

Thanks but Oooops ... I checkmarked the wrong post . I meant to checkmark post #14 not post #17 ... can anything be done about that?

 

[robert600]

oh ... i think i might have figured it out ... just click on the checkmark again?

 

[hotaru.hino]

YES!!! I just completed a small test of this and it worked perfectly. Thanks so much! It makes me wonder why "use database file to detect change" isn't enabled by default?

Probably because it requires a file to live in the directories being touched, which isn't ideal for all situations.

For example, I don't need this file living in my cloud backups.

 

[robert600]

Probably because it requires a file to live in the directories being touched, which isn't ideal for all situations.

For example, I don't need this file living in my cloud backups.

Ah .... ok, that makes sense. Having it enabled is a game changer for me though ... easy enough, if you know how ... which I didn't! lol

 

[MWink64]

YES!!! I just completed a small test of this and it worked perfectly. Thanks so much! It makes me wonder why "use database file to detect change" isn't enabled by default?

It is the default in Two Way and Update mode, just not Mirror.