What do I do with a full hard drive?

[mirzai_m]

Hey everyone,

OK, learning time for me. I have to admit something (no flaming please)....I don't back up my data. AAARRRGHH there I said it, and I hate that I don't do it. I want to become a better person, and I need your help.

How...do I back up my data?

I have a 200GB HD full of pics of the kids and I'm afraid this thing's gonna fail just to teach me a lesson.

Do I burn my data to DVD's and stash them?
Do I install a new hard drive and store the old one?
Do I install a second hard drive?
External drive?
What do I do with a full hard drive?

Also, I'm building a new rig and was wondering if (to avoid this issue) I should set up a RAID 1 with (2) 1TB WD Black Caviars.

I'm just not sure what the "end game" for archiving data while backing it up for safe keeping is.

Please help me be good at this.

Mike

 

[jedimasterben]

The safest thing to do is to have it on multiple types of media in different locations. What I do, personally, is save to my new storage drive, archive to DVD, and the really important stuff gets put on a separate computer that is mostly off and not connected to my network.

RAID 1 is good, but will only be recoverable on that same computer on the same RAID ports, so if your motherboard fries, you may be FUBAR. Offsite backup is the best, just remember to continue backing up!

 

[Paperdoc]

You can run the numbers to make your own decision, but I'll outline what many conclude. If you look at various ways to make and renew backups, even of files (such as photos) that do NOT change over time, you can consider storing them on extra hard drive(s), on a stack of CD-R's (usually 'way too many CD's required), on a stack of DVD-R's (much higher capacity per disk), on tapes (requires a stock of tapes AND a backup tape drive), or on a remote web server that rents you space for your backups. Now, at just under 5 GB per disk on a DVD-R, you would need to buy, burn and store over 40 disks for your current collection. That will take you about 20 minutes per disk to do the burning, and you have to be there to keep changing disks. For photos you probably do not need to replace older disks with updated ones because the files are not changing over time, but other files do need updating. For those latter people might well use DVD-RW disks (more expensive). CD-R's and CD-RW's simply don't have the capacity to make such a system viable.

Two other items regarding DVD systems need consideration. What happens when your collected files head for 1 TB? Now your DVD stack starts to look like over 200 disks, and the time to make a complete backup approaches 70 hours with NO attempt to read them and verify that they are valid backups!

A tape-based system is just ideally suited to this task, especially the ones with multi-tape changers built in. But the cost of the hardware and the cost of the tapes usually is much too high for most home users. Professional server systems use them because they do the job very well with minimal staff intervention (and labor cost) and these shops do LOTS of backups and tape verifications to be sure it is done right and they never risk losing data.

A web-based system can be really good if the price is right, especially for data that does change, requiring backup files to be updated frequently. You just need the cash and a we connection with high bandwidth to make the process convenient. then you need to be confident the company will stay in business for a long time.

Many home users find a second (or more) hard drive the best option. In this case, the best option is a stand-alone external hard drive. You connect it to your computer, make the backup of your entire system onto one high-capacity unit, verify the backup, then disconnect it for safety. Ideally you then store it off-site. Ideally, also, you get two of these units and alternate between them. Today you can buy 1.0 TB external HDD units for under $200, similar to the cost of 100 DVD-RW blank disks and much less time-consuming for the operations.

For all of these options, two important considerations are the longevity of the storage medium, and the pace of change of technology. There are plenty of people who will advise that a DVD-RW disk cannot be relied upon to keep its data perfect and error-free for more than (stick in a number - maybe 5??) years. So, as a means of making one archive of files that never will have to be changed and updated, this still has a limit - how long can you be sure they will give you back your data? Similar arguments can apply to hard drives, although they won't mechanically wear out if they are used only infrequently. Then there is the question: if I make a collection of unchanging files on (pick your medium) today, will I be able to buy and use a reader for that in 10 years, or will all those reader / writer units be off the market by then? This means you will have to plan to update (that is, change!) your whole plan periodically. So the money you invest now only buys you reliable archives for limited years.

Bottom line, you have to do enough homework to make your own decisions on cost and convenience as well as reliability, and there is no permanent answer.

IMPORTANT: RAID1 is great for keeping running even when a HDD unit fails. BUT it is NOT a substitute for a backup system, so don't use it that way. There are so many ways for data to be lost or whole systems to fail when your only storage system is hard drives in your main computer case. Get and USE a backup system.

 

[mirzai_m]

You can run the numbers to make your own decision, but I'll outline what many conclude. If you look at various ways to make and renew backups, even of files (such as photos) that do NOT change over time, you can consider storing them on extra hard drive(s), on a stack of CD-R's (usually 'way too many CD's required), on a stack of DVD-R's (much higher capacity per disk), on tapes (requires a stock of tapes AND a backup tape drive), or on a remote web server that rents you space for your backups. Now, at just under 5 GB per disk on a DVD-R, you would need to buy, burn and store over 40 disks for your current collection. That will take you about 20 minutes per disk to do the burning, and you have to be there to keep changing disks. For photos you probably do not need to replace older disks with updated ones because the files are not changing over time, but other files do need updating. For those latter people might well use DVD-RW disks (more expensive). CD-R's and CD-RW's simply don't have the capacity to make such a system viable.

Two other items regarding DVD systems need consideration. What happens when your collected files head for 1 TB? Now your DVD stack starts to look like over 200 disks, and the time to make a complete backup approaches 70 hours with NO attempt to read them and verify that they are valid backups!

A tape-based system is just ideally suited to this task, especially the ones with multi-tape changers built in. But the cost of the hardware and the cost of the tapes usually is much too high for most home users. Professional server systems use them because they do the job very well with minimal staff intervention (and labor cost) and these shops do LOTS of backups and tape verifications to be sure it is done right and they never risk losing data.

A web-based system can be really good if the price is right, especially for data that does change, requiring backup files to be updated frequently. You just need the cash and a we connection with high bandwidth to make the process convenient. then you need to be confident the company will stay in business for a long time.

Many home users find a second (or more) hard drive the best option. In this case, the best option is a stand-alone external hard drive. You connect it to your computer, make the backup of your entire system onto one high-capacity unit, verify the backup, then disconnect it for safety. Ideally you then store it off-site. Ideally, also, you get two of these units and alternate between them. Today you can buy 1.0 TB external HDD units for under $200, similar to the cost of 100 DVD-RW blank disks and much less time-consuming for the operations.

For all of these options, two important considerations are the longevity of the storage medium, and the pace of change of technology. There are plenty of people who will advise that a DVD-RW disk cannot be relied upon to keep its data perfect and error-free for more than (stick in a number - maybe 5??) years. So, as a means of making one archive of files that never will have to be changed and updated, this still has a limit - how long can you be sure they will give you back your data? Similar arguments can apply to hard drives, although they won't mechanically wear out if they are used only infrequently. Then there is the question: if I make a collection of unchanging files on (pick your medium) today, will I be able to buy and use a reader for that in 10 years, or will all those reader / writer units be off the market by then? This means you will have to plan to update (that is, change!) your whole plan periodically. So the money you invest now only buys you reliable archives for limited years.

Bottom line, you have to do enough homework to make your own decisions on cost and convenience as well as reliability, and there is no permanent answer.

IMPORTANT: RAID1 is great for keeping running even when a HDD unit fails. BUT it is NOT a substitute for a backup system, so don't use it that way. There are so many ways for data to be lost or whole systems to fail when your only storage system is hard drives in your main computer case. Get and USE a backup system.

Thanks for taking the time, that's what I had considered but thought I was "overthinking it". Now I don't think I am, it's a real task!

 

[Pailin]

Very good advice.

(please bear with my lack of use of correct terminology - was a long time ago I read this and no time tonight to re-research it + is not mega easy to find)

But I should add that the data on a HDD used for storage - especially on newer HDD's with their high data densities using Perpendicular Magnetic Recording (PMR) - needs to be "refreshed" every few years.

Older HDD's did not suffer as badly as modern ones as the spots of data were not so tightly packed together.

This is due to magnetic "bleed" between the data pits. I understand new technologies try to reduce this, but as far as I know it is still an issue. But basically the higher the data density the harder it is to stop this from happening.

Recopying the data to the drive sorts it every 2 - 5 years or so is recomended to be safe.
A defrag that moves All the data also works.