Question How Much Should You Back Up Your Data and What is the Likelihood of Drive Failure?

[avrona]

So I've been doing some research into how much people backup their data and what the odds are of their backups actually coming in handy, as I've never really understood the whole backup culture that exists in the tech world. I have no backups of any of my files and thankfully I've never needed one, so I decided to do some research and make a video on the subject. So I just want to know, if you are aware of the percentages and likelihoods of different types of drives failing, please do let me know, and also let me know if you backup your data, if you do, what kind of data is it, and how much do you recommend both normal PC users and more advance users backup their data?

 

[jay32267]

I have lots of PCs in a plant that I've been responsible for for about 30 years so I think I have at least an idea regarding drive failures.

I find that HDDs fail pretty regularly. I'd say 5 years is about average. I've had some fail much sooner...and some last longer than 10....maybe even 15 years.

I find SSDs fail less.....although I have not been running SSDs nearly as long as HDDs so I don't have "long term experience".

As far as backups.....I run 3 systems at home and I clone each main HD (the one with the OS) about once a month. That way I can always get back in case of a failure or a virus etc.

 

[ss202sl]

Drives WILL fail. Many get replaced before they fail, but none of them will last forever. The amount of backups are directly related to how mch the data changes, and how important the data is. It work we have some critical systems that get backed up several times a day while others are only backed up weekly. Can't provide a solid number, but we do restore files every week. Most of the file restores are caused by deletions and user error, not from failed hard drives.

 

[hang-the-9]

So I've been doing some research into how much people backup their data and what the odds are of their backups actually coming in handy, as I've never really understood the whole backup culture that exists in the tech world. I have no backups of any of my files and thankfully I've never needed one, so I decided to do some research and make a video on the subject. So I just want to know, if you are aware of the percentages and likelihoods of different types of drives failing, please do let me know, and also let me know if you backup your data, if you do, what kind of data is it, and how much do you recommend both normal PC users and more advance users backup their data?

Chance of drive failure, as far as anyone should be treating their data, is 100%. It's just a matter of when.
You wear a seatbelt? Wash your hands after getting them dirty? Keep a spare tire and jumper cables in the car? Try to avoid mosquitos if there is a virus? Don't smoke because it increases chance of cancer? Have health, car, home insurance? Making data backups is all in the same category as all of those.

While you may never get a drive failure, why risk it? Drives fail, that is a fact. You don't k now when they will fail, and worrying about data after a drive fails is too late. So the logical and reasonable thing to do is treat it like it WILL fail, and back up data. You don't start checking your parachute after you jump out of a plane. 99% of issues we see on the forums having to do with data loss will not be issues if people did backups. Every day there are a dozen posts "I lost my very important data, help!". If they took 2 minutes to copy that important data a week ago it's not an issue.

The general recommendation is to have a base system image with your programs and settings, then do a backup of your files, and keep things up-dated. Best practice is to keep more than one backup location and media.

If you are Bill Gates or Joe BillyBob from the farm does not matter, there is no difference between tech people backups and "normal" people backups. What you need backed up is what you need backed up, only difference may be number of files you have.

 

[avrona]

Chance of drive failure, as far as anyone should be treating their data, is 100%. It's just a matter of when.
You wear a seatbelt? Wash your hands after getting them dirty? Keep a spare tire and jumper cables in the car? Try to avoid mosquitos if there is a virus? Don't smoke because it increases chance of cancer? Have health, car, home insurance? Making data backups is all in the same category as all of those.

While you may never get a drive failure, why risk it? Drives fail, that is a fact. You don't k now when they will fail, and worrying about data after a drive fails is too late. So the logical and reasonable thing to do is treat it like it WILL fail, and back up data. You don't start checking your parachute after you jump out of a plane. 99% of issues we see on the forums having to do with data loss will not be issues if people did backups. Every day there are a dozen posts "I lost my very important data, help!". If they took 2 minutes to copy that important data a week ago it's not an issue.

The general recommendation is to have a base system image with your programs and settings, then do a backup of your files, and keep things up-dated. Best practice is to keep more than one backup location and media.

If you are Bill Gates or Joe BillyBob from the farm does not matter, there is no difference between tech people backups and "normal" people backups. What you need backed up is what you need backed up, only difference may be number of files you have.

There's also a lot of other horrible things that can happen to use, but they are as unlikely to happen as drives failing, so we don't prepare for them, so what's the difference with data? All I have is what I like to call "unintentional backups", which just so happen to all be for my most important files, but in none of these scenarios a backup was my goal. Since the SD cards I have for my camera are rather big I can keep a lot of photos on them, so instead of moving them over to my PC I copy them so if I ever need them on the camera they are there. I also have some files store in the cloud, not for backup's sake, but rather because they are files I need to have access to from other places. I also have a lot of important channel files, all of my assets, video clips, finished and ongoing videos are all in one folder, but recently I got a pcie 4.0 NVMe so I decided to just copy over the files from my hard drive to there to speed things up.

 

[TJ Hooker]

There's also a lot of other horrible things that can happen to use, but they are as unlikely to happen as drives failing, so we don't prepare for them, so what's the difference with data?

You balance likelihood of a negative outcome occurring, the cost of that outcome, and the cost (in time and/or money) of preparing for that outcome. The cost of backups is quite small, so if you value your data at all why would you not?

 

[USAFRet]

What is your backup situation at home?

What is your backup situation at home? And if you don't do that, why not? Every single day, I read multiple threads here of "How do I get my stuff back?" or "That drive had 5 years of photos of my kids!!" Be it a dead drive, dropped phone, virus, accidental deletion, formatting the wrong...

forums.tomshardware.com

Drives WILL fail.
Or you'll get a nasty virus
Or ransomware
Or the external USB drive will fall off the desk
Or you'll accidentally delete something
Or a lightning strike near your house will fry everything

Physical drive fail is not the only way to lose data.

Having said that, I had exactly that happen last December. 960GB SAnDisk SSD just died. Suddenly, without warning.
Nothing I could do would bring it back to life.
Why did it die? Don't know. But mostly...don't care.
It was dead.

Swap in a new drive, click click...about an hour later, all 605GB data that was on it recovered 100%, exactly as it was at 4AM that morning.
My backup routine in the above link justified its time and cost wit htat one recovery.

 

[JBHapgood]

So I've been doing some research into how much people backup their data and what the odds are of their backups actually coming in handy, as I've never really understood the whole backup culture that exists in the tech world. I have no backups of any of my files and thankfully I've never needed one, so I decided to do some research and make a video on the subject. So I just want to know, if you are aware of the percentages and likelihoods of different types of drives failing, please do let me know, and also let me know if you backup your data, if you do, what kind of data is it, and how much do you recommend both normal PC users and more advance users backup their data?

The likelihood of a mechanical hard drive failing is 100%. It's just a matter of when. It could be tomorrow, or five years from now. But it will happen.

Solid state drives should be more reliable, as they have no moving parts. But there are limits to the number of times the components on the chip can be rewritten. The on-board controllers can work around worn-out memory locations, but eventually the drive will run out of them. And before that there may be a catastrophic failure of one or more of the chips on the drive. Or a power surge or a virus might destroy your data.

You always need to back up your data. How you do that is up to you, based on your own needs.

I have a two-level backup scheme. I have a local external USB 3 drive that is turned off most of the time, except for when I'm doing a backup. I also have a cloud backup subscription. And some files also get backed up to an optical drive.

The system partition (C: ) is on a solid state drive. I back it up as needed to the local external drive (usually monthly) with disk-imaging software that can also extract individual files from the image. Because the partition contains applications and system software, it doesn't change much. So frequent backups aren't needed.

I have three data partitions. One of them shares the solid-state drive with the system partition, the other two are on a mechanical hard drive. I have written a program that runs on Take Command (a much more powerful replacement for the Windows command prompt). It can incrementally back up any or all of the data partitions, writing only changed files to the local external drive. I use it as needed, at least weekly. If I've made a bunch of changes (as with photography or music files I'm working on) I run it on that partition. The incremental backup doesn't take long. The cloud backup service also makes daily incremental backups of selected folders on the data partition.

There's no one "correct" backup scheme. But you must back up any data that you wouldn't mind losing to a crash, power surge, virus, or stupid "user error."

 

[hang-the-9]

There's also a lot of other horrible things that can happen to use, but they are as unlikely to happen as drives failing, so we don't prepare for them, so what's the difference with data? All I have is what I like to call "unintentional backups", which just so happen to all be for my most important files, but in none of these scenarios a backup was my goal. Since the SD cards I have for my camera are rather big I can keep a lot of photos on them, so instead of moving them over to my PC I copy them so if I ever need them on the camera they are there. I also have some files store in the cloud, not for backup's sake, but rather because they are files I need to have access to from other places. I also have a lot of important channel files, all of my assets, video clips, finished and ongoing videos are all in one folder, but recently I got a pcie 4.0 NVMe so I decided to just copy over the files from my hard drive to there to speed things up.

Well if you are looking for someone to tell you that you don't need backups, don't do any. The answer is "do backups". If you want to give reasons as to why not to do backups, because most of the time disks are OK, that is up to you. And prudent people try to prevent horrible things, you don't want to be fat and die of a heart-attack at 50, eat good and exercise. You don't want to die in a fire, people have grates around their fireplaces and don't throw gas cans in them, you don't want your plane crashing, you build it well and do many tests and have redundant parts in case one fails, etc... People prepare for pretty much anything possible, either actively or by just not being idiots. You don't prepare for not being eaten by a Lion, but at the same time it's not a smart idea to make that happen by climbing into a cage with one at the zoo or running up to one waving your arms when on a safari.

As I said, do you want to start checking if your parachute is good after or before you jump from the plane? If you are OK with checking it after, then you are OK, go to sleep feeling good about your data as it is and deal with any issues after they come up.

I know people that smoke and use the excuse because someone they knew lived to be 90 and they smoked so smoking is fine for your health. That is pretty much what you are saying. If it's only a small chance, why waste my time with backups. Well, that is fine to say, right up till your drive fails and then you are posting "help me recover my data". This is probably why we need warnings on our cups that things may be hot or signs at zoos saying "don't climb in to pet the lions", no-one thinks anything will happen to them, it's always the other people.

 

[USAFRet]

No backup? You could be like this guy:

[SOLVED] - Cloning dying HDD question

So my HDD is dying violently. One day, I got SMART error on my drive, the next day it told me something was wrong, that chkdsk could fix. After that all went to hell and I lost my file systems and so on. After all of this I ran another program, which told me I have a hellalot of bad sectors, so...

forums.tomshardware.com

 

[fzabkar]

@avrona, could we see SMART reports for all your HDDs and SSDs?

https://crystalmark.info/en/software/crystaldiskinfo/

 

[avrona]

@avrona, could we see SMART reports for all your HDDs and SSDs?

https://crystalmark.info/en/software/crystaldiskinfo/

Do already have crystaldiskinfo, what exactly is the SMART report though?

No backup? You could be like this guy:

[SOLVED] - Cloning dying HDD question

So my HDD is dying violently. One day, I got SMART error on my drive, the next day it told me something was wrong, that chkdsk could fix. After that all went to hell and I lost my file systems and so on. After all of this I ran another program, which told me I have a hellalot of bad sectors, so...

forums.tomshardware.com

No, no backups, because again they seem kinda pointless.

 

[USAFRet]

No, no backups, because again they seem kinda pointless.

Pointless, right up until the moment your drive dies. I had exactly that happen a few months ago. Poof, dead.
Or you get a nasty virus.
Or ransomware.

But, whatever works for you.
If you have nothing on it you wish to keep...feel free to ignore any backup recommendations.
If you just treat your PC as a fancy playstation, then a 'backup' may be too much work.

Me? I'd be pretty pissed if I lost a pic of my eldest grandson, at age 2 (now 14), in a pink tutu.
Or the multiple copies of my resume.
Or the several databases I have in work.
Or my budget xls.
Or...


People don't see the need for one. Until 5 minutes after they do.

 

[fzabkar]

Do already have crystaldiskinfo, what exactly is the SMART report though?

Run CDI and find out.

 

[avrona]

Run CDI and find out.

https://imgur.com/a/gV0KBnk

View: https://imgur.com/a/gV0KBnk

 

[USAFRet]

That report just means that the physical drive probably won't die in the next 5 minutes.

 

[geofelt]

Hard drive failure rates are hard to come by.
Here is one report I found:

Backblaze Drive Stats: 2018 Hard Drive Failure Rates

Backblaze has 99,636 spinning hard drives. This review looks at the quarterly and lifetime statistics for the data drive models in operation in our data centers.

www.backblaze.com

No matter.
If you have files on your pc that you care about, then you should have a plan for External backup.

 

[fzabkar]

Both Seagates are reporting non-zero raw values for the Command Timeout attribute. That's often a sign that the drive is struggling to read certain sectors.

Also, the Crucial SSD has a large number of Unexpected Power Losses. Unless it has power loss protection (ie a bank of capacitors), I would be concerned about this attribute.

 

[avrona]

Both Seagates are reporting non-zero raw values for the Command Timeout attribute. That's often a sign that the drive is struggling to read certain sectors.

Also, the Crucial SSD has a large number of Unexpected Power Losses. Unless it has power loss protection (ie a bank of capacitors), I would be concerned about this attribute.

Well it's been serving me as my main drive since 2013, so could that be anything to do with it?

 

[USAFRet]

Well it's been serving me as my main drive since 2013, so could that be anything to do with it?

Ticking time bomb.

Not sure what you want any of us out here to say.

"Go ahead, never ever bother with backing up your data. Nothing bad will ever happen."
And then...you delete something by accident. Or that drive actually does die. Or your house gets hit by lightning. Or you click the wrong thing and get a nasty virus.

"But...those guys told me I had nothing to worry about!! I lost all my stuff! They suck!!"

Your data is your data. It has zero value to us out here.
It may or may not be valuable to you. Even in the trivialities of time taken to rebuild your system after something bad happens.

Drives die. All of them, eventually. Maybe today, maybe tomorrow, maybe not until 2025.
But it will surely die. And that's just the physical drive fail.
There are many other ways to 'lose your stuff'.

But, it's your stuff.
Do as you will.

 

[fzabkar]

The ST1000DM003 is a member of the same family as the infamous ST3000DM001 which was the subject of a law suit. ISTR that BackBlaze recorded a 30% failure rate for the latter.

BTW, you will probably find that Seagate's firmware update will fail. The last time I looked, Seagate still hadn't fixed the bug which I reported several years ago. In fact I provided the bug fix and a detailed explanation, but nobody cared.

As for unexpected power losses, this means that the SSD's firmware does not always have enough time to complete its "housekeeping" tasks before shutdown. A common result is failure to power up at the next power cycle.

SSDs do wear levelling. This means that they constantly reassign new physical blocks to the same LBA. In order to keep track of this, the firmware maintains a Flash Translation Layer (FTL). The FTL associates each LBA with its corresponding physical NAND block. If the FTL becomes corrupted, then the firmware goes into panic mode and the data become inaccessible.

 

[mdd1963]

Individual users will have to decide what data (if any) really needs to be 'backed up'...

An OS and all favorite applications/utilities can be redone from scratch within a couple hours, given a decent connection speed. (If only Ninite had everything I used!)

If you have no files/photos on your computer's drive that are important enough to be stored twice, and/or have decided that your system is impervious to ransomware or Windows updates that brick your system from booting, that is indeed your choice, and a fine choice...for you. (With the plethora of free cloud storage accounts with Mega, Dropbox, GoogleDrive, OneDrive, PCloud, Jottacloud, etc, it'd be hard to lose truly priceless photos/docs....)

As the above Gigabyte drive's CDI info output screen cap is clearly from a PCI-e 4.0 /M.2 drive on an X570 board, given their 7-8 weeks of total impressive history of established reliability, I'm sure nothing could ever go wrong with them in the future! Perfect place for a bitcoin wallet, ....preferably in RAID 0!

 

[x01x]

I've been backing up my photos and videos for the last 20 years.
At first I used CD-Rs, then DVD-Rs.
Now I back up a complete copy of everything on an external HDD + BD-R + Cloud storage.

 

[hang-the-9]

Do already have crystaldiskinfo, what exactly is the SMART report though?

No, no backups, because again they seem kinda pointless.

Well, if you are looking for someone to tell you "don't do backups", that won't happen. It's your data, no-one aside from you will have issues if it's lost. It's a very short answer, best idea is to backup anything you don't want to lose. There is no discussion about it as to how often drives may fail, you won't know, we won't know. If you are no prepared for when it does happen, then there are consequences. Just because most people don't shoot themselves when cleaning their gun does not mean that you should not check to see if yours is loaded. And if you don't and shoot yourself, you will know why it happened, because you did not follow correct procedures.

 

[avrona]

Why is it the "correct procedure"? What dictates you should backup drives despite the extremely low odds? And I do actually have some of my more important files "backed up" but it's through what I like to call "unintentional backups", when you are doing something resulting in a backup where it wasn't your primary goal, it's especially good as files you most often do "unintentional backups" of are you most important files, and it protect me from my only data loss I've experienced.