Question Can a PNG file with defective bit still be opened ?

[Oblivion77]

Dear all

If a PNG file has 1 or more bit, that is either unreadable, corrupt or broken.

What happens, if the file is opened in MS Paint or another software?
Would an error message appear saying something / file uable to open?
Would the software try to repair or fill the gap?
Is it the same for NTFS and FAT34?

Thank you for replying

Best regards

 

[USAFRet]

And getting back to the original question: "Can a PNG file with defective bit still be opened ?"

Yes and no, as seen in Post #17


Change the value of a bit may change the value of the byte it is a part of.

A single changed byte can render a PNG openable, but not fully readable.
Or, if that byte were changed in a different part of the file....completely unopenable as a .PNG.

 

[Oblivion77]

It still don't understand, how the drive can detect an error and correct, it requires the drive to compare the corrupt file with a flawless version of the file?

It doesn't need an exact copy of the data to correct it. It uses error correcting code to detect and fix errors.

1.
How can it correct the flipped bits, if it only knows that there is supposed to be X 0 and X 1? But not where the 1's and 0's are located in the data bytes?

2.
If the drive determines some bits are flipped and corrects them, does the drive then change the original stored file?
If yes, does it then update the "Last modified" data in the file metadata?
Example:
File A stored 01.01.2022
Something occurs and bits in File A is somehow flipped 01.02.2022
The drive detects the flipped bits in File A and the bits are corrected 01.03.2022

 

[Oblivion77]

When a sector is written to the drive, the drive calculates the ECC bytes and stores them together with the data bytes. When the drive reads back the sector, it recalculates the ECC. If the new and original ECC bytes don't match, then the drive knows that there was a read error, either in the data bytes or in the ECC bytes. If the number of flipped bits is not too great, the drive is able to determine which bits were flipped. It then automatically and transparently corrects the data before passing it to the host. Otherwise, if the number of flipped bits exceeds the strength of the error correction, the drive returns a read error.

1.
How can it correct the flipped bits, if it only knows that there is supposed to be X 0 and X 1? But not where the 1's and 0's are located in the data bytes?

2.
If the drive determines some bits are flipped and corrects them, does the drive then change the original stored file?
If yes, does it then update the "Last modified" data in the file metadata?
Example:
File A stored 01.01.2022
Something occurs and bits in File A is somehow flipped 01.02.2022
The drive detects the flipped bits in File A and bits are corrected 01.03.2022

 

[Oblivion77]

File A on local drive
Copy file A to USB drive (B)
Create whole file again from the bottom (C)

Any time the data is in flight, anything can happen to it. If the computer is in an environment that's hostile to electronics, it could easily develop errors along the way.

How would you make sure, that the 3 files are identical?

 

[fzabkar]

1.
How can it correct the flipped bits, if it only knows that there is supposed to be X 0 and X 1? But not where the 1's and 0's are located in the data bytes?

2.
If the drive determines some bits are flipped and corrects them, does the drive then change the original stored file?
If yes, does it then update the "Last modified" data in the file metadata?
Example:
File A stored 01.01.2022
Something occurs and bits in File A is somehow flipped 01.02.2022
The drive detects the flipped bits in File A and bits are corrected 01.03.2022

The drive knows nothing about files. It only knows whether the data in a particular sector is good or bad. Moreover, the host computer has no way of knowing that a sector has been corrected. Therefore there is no change to the date/time stamp.

The ECC algorithm is constructed in such a way that it can detect which bits are bad, and it then corrects them. Depending on the firmware, the drive may correct a bad sector by rewriting it with the corrected data.

 

[hotaru.hino]

1.
How can it correct the flipped bits, if it only knows that there is supposed to be X 0 and X 1? But not where the 1's and 0's are located in the data bytes?

2.
If the drive determines some bits are flipped and corrects them, does the drive then change the original stored file?
If yes, does it then update the "Last modified" data in the file metadata?
Example:
File A stored 01.01.2022
Something occurs and bits in File A is somehow flipped 01.02.2022
The drive detects the flipped bits in File A and the bits are corrected 01.03.2022

Any bits that are determined to be incorrect are fixed transparently. Also storage drives do not know what a file is. The only thing a storage drive knows about and manages are chunks of bytes called sectors.

How would you make sure, that the 3 files are identical?

If you wanted to absolutely be sure that the 3 files are as identical as possible, take apart the drive, find the physical location of the bits where the data is stored, and poke at what their values are (be it magnetic flux for hard drives or voltage for flash media).

Or you can just run it through something like an MD5, SHA, or whatever hash function that gives a sufficiently large digest to satisfy your paranoia that the chances of a collision happening are basically 0 and verify the output is the same.

 

[Oblivion77]

If you wanted to absolutely be sure that the 3 files are as identical as possible, take apart the drive, find the physical location of the bits where the data is stored, and poke at what their values are (be it magnetic flux for hard drives or voltage for flash media).

Or you can just run it through something like an MD5, SHA, or whatever hash function that gives a sufficiently large digest to satisfy your paranoia that the chances of a collision happening are basically 0 and verify the output is the same.

1.
Is the command fc /b as efficient / good as MD5 or SHA?

2.
"From when the data is picked up from the drive to the time it arrives in RAM and processed, anything could've happened"
Does that include when running the command fc/ b or MD5 or SHA?

3.
When running the command fc/ b or MD5 or SHA, could they trigger bits being corrected by the drive, before showing the result?
Run fc /b on file A and B
Drive determines there are flipped bits in file A
Drive correct bits in file A
The result shows No differences encountered with file A and B

 

[Oblivion77]

Any bits that are determined to be incorrect are fixed transparently. Also storage drives do not know what a file is. The only thing a storage drive knows about and manages are chunks of bytes called sectors.

I will watch the video, as soon as I can. Thank you for posting it.

If bits has been flipped due to bad RAM.
1.
When would the bits get flipped (if they did), the first time the file is written to the drive?

2.
When does the bits get flipped back / corrected? The first time the file is read, after it has been written to disk?

3.
When the bits are first time corrected on the drive, then its permanent?

 

[Oblivion77]

The drive knows nothing about files. It only knows whether the data in a particular sector is good or bad. Moreover, the host computer has no way of knowing that a sector has been corrected. Therefore there is no change to the date/time stamp.

The ECC algorithm is constructed in such a way that it can detect which bits are bad, and it then corrects them. Depending on the firmware, the drive may correct a bad sector by rewriting it with the corrected data.

If bits has been flipped due to bad RAM.
1.
When would the bits get flipped (if they did), the first time the file is written to the drive?

2.
When does the bits get flipped back / corrected? The first time the file is read, after it has been written to disk?

3.
When the bits are first time corrected on the drive, then its permanent?

 

[fzabkar]

FC /B executes a direct byte-by-byte comparison between two files. It is better than any hash.

Bad bits in RAM can flip at any time, either during writing, or subsequently during reading.

Any bad bits are corrected during reading -- obviously.

Corrections are not permanent. Bits can flip again.

 

[hotaru.hino]

1. When would the bits get flipped (if they did), the first time the file is written to the drive?

2. When does the bits get flipped back / corrected? The first time the file is read, after it has been written to disk?

3. When the bits are first time corrected on the drive, then its permanent?

1. It can happen at any time for any reason
2. Whenever the data is accessed
3. Bits can still be flipped, again at any time for any reason.

 

[Oblivion77]

FC /B executes a direct byte-by-byte comparison between two files. It is better than any hash.

Bad bits in RAM can flip at any time, either during writing, or subsequently during reading.

Any bad bits are corrected during reading -- obviously.

Corrections are not permanent. Bits can flip again.

When running the command fc /b, the drive access the data / file.
1.
"From when the data is picked up from the drive to the time it arrives in RAM and processed, anything could've happened"
Does that include when running the command fc/ b?

2.
When running the command fc/ b, could that trigger bits being corrected by the drive, before showing the result?

3.
Would you say that, when bits somehow gets flipped, it is the same as the file being changed?

 

[Oblivion77]

1. It can happen at any time for any reason
2. Whenever the data is accessed
3. Bits can still be flipped, again at any time for any reason.

When running the command fc /b, the drive access the data / file.
1.
"From when the data is picked up from the drive to the time it arrives in RAM and processed, anything could've happened"
Does that include when running the command fc/ b?

2.
When running the command fc/ b, could that trigger bits being corrected by the drive, before showing the result?

3.
Would you say that, when bits somehow gets flipped, it is the same as the file being changed?

 

[Oblivion77]

07:53 - Keep in though, there is no method for error detection, or correction, that could give you 100% confidence that the message you receive is the one the sender intended. After all, enough random noise can always change one valid message into another valid message just by pure chance.
1.
That gets solved when using more than 1 parity bit?

Instead, the goal is to come up with a scheme that is robust up to a certain maximum number of errors
2.
What happens when the maximum number of errors is exceeded?
Read error?
Partially open file?

3.
Are all other error detection codes as effective / more effective, than hamming code?

4.
What happens when more than 1 parity bit gets flipped? The code is still viable?

 

[Oblivion77]

FC /B executes a direct byte-by-byte comparison between two files. It is better than any hash.

Bad bits in RAM can flip at any time, either during writing, or subsequently during reading.

Any bad bits are corrected during reading -- obviously.

Corrections are not permanent. Bits can flip again.

Dear fzabkar


Thank you so much for all your answers so far. I hope, that you will continue to try to answer the following:

When running the command fc /b, the drive access the data / file.
1.
"From when the data is picked up from the drive to the time it arrives in RAM and processed, anything could've happened"
Does that include when running the command fc/ b?

2.
When running the command fc/ b, could that trigger bits being corrected by the drive, before showing the result?

3.
Would you say that, when bits somehow gets flipped, it is the same as the file being changed?

4.
Why don’t ECC correct all flipped bits, when a file is opened with errors / partially opened?
It’s only effective up to X number of bits / threshold?

 

[Oblivion77]

1. It can happen at any time for any reason
2. Whenever the data is accessed
3. Bits can still be flipped, again at any time for any reason.

Dear hotaru

Thank you so much for all your answers so far. I hope, that you will continue to try to answer the following:

When running the command fc /b, the drive access the data / file.
1.
"From when the data is picked up from the drive to the time it arrives in RAM and processed, anything could've happened"
Does that include when running the command fc/ b?

2.
When running the command fc/ b, could that trigger bits being corrected by the drive, before showing the result?

3.
Would you say that, when bits somehow gets flipped, it is the same as the file being changed?

4.
Why don’t ECC correct all flipped bits, when a file is opened with errors / partially opened?
It’s only effective up to X number of bits / threshold?

 

[fzabkar]

1/ Yes
2/ Yes
3/ Yes
4/ ECC algorithms append extra bits to the data. If there are more ECC bits, then this makes the algorithm stronger and it is able to correct more bit errors. The video posted by hotaru.hino should answer this question.

 

[hotaru.hino]

I'm curious, is the goal here to figure out a 100% fool-proof method to avoid data corruption? Or are you just digging down the rabbit hole of how data corruption happens?

I feel like we're going around in circles here.

 

[Oblivion77]

I'm curious, is the goal here to figure out a 100% fool-proof method to avoid data corruption? Or are you just digging down the rabbit hole of how data corruption happens?

I feel like we're going around in circles here.

I find this topic really interesting, and when I go about my day, I think about your answers - and then new questions pop-up. If you do not like answering any more questions, I will not bother you anymore. But thank you, so far.

 

[Oblivion77]

1/ Yes
2/ Yes
3/ Yes
4/ ECC algorithms append extra bits to the data. If there are more ECC bits, then this makes the algorithm stronger and it is able to correct more bit errors. The video posted by hotaru.hino should answer this question.

1.
The appended extra bits used in ECC, they are created from the file bits
If the file bits are corrected later, it is from the appended bits
True?

2.
Theoretically, the appended bits are equal the file bits / could be used to create the file bits?

3.
Can the appended bits be corrected through ECC, or are they always “locked”?

 

[Karadjgne]

Define 'corrupted'. If the original 'save' created a perfectly black picture, with a single white pixel, that'd be a corruption if all pixels were supposed to be black. Copy that to a different file or location and it's no longer 'corrupted' because it is now a perfect copy of the original. Doesn't matter if that single white pixel remains, it got copied exactly as was.

You might view it as corrupted, the pc won't. The copy is exactly perfect. Which makes it hard to 'fix' as there's nothing to fix as the copy is perfect. Meaning you'd need software like mspaint to go in and change the copy, change the valid white pixel to a valid black pixel so that your perception is now correct and 'uncorrupted'.

 

[Oblivion77]

Define 'corrupted'. If the original 'save' created a perfectly black picture, with a single white pixel, that'd be a corruption if all pixels were supposed to be black. Copy that to a different file or location and it's no longer 'corrupted' because it is now a perfect copy of the original. Doesn't matter if that single white pixel remains, it got copied exactly as was.

You might view it as corrupted, the pc won't. The copy is exactly perfect. Which makes it hard to 'fix' as there's nothing to fix as the copy is perfect. Meaning you'd need software like mspaint to go in and change the copy, change the valid white pixel to a valid black pixel so that your perception is now correct and 'uncorrupted'.

Thank you for your reply

Corrupted =
What I mean: The file bits are flipped, no longer readable / destroyed etc.
A more widespred explanation: "a file that has become damaged and refuses to open properly"

"If the original 'save' created a perfectly black picture, with a single white pixel, that'd be a corruption if all pixels were supposed to be black"
That corruption whould happen in ram?

"Copy that to a different file or location and it's no longer 'corrupted' because it is now a perfect copy of the original."
Why would ECC not correct this, correct the bits and make the white pixel black again?

What you mean is, unless myself or ECC corrects the corruption, the corruption would persist when copying?

 

[Oblivion77]

1/ Yes
2/ Yes
3/ Yes
4/ ECC algorithms append extra bits to the data. If there are more ECC bits, then this makes the algorithm stronger and it is able to correct more bit errors. The video posted by hotaru.hino should answer this question.

Dear fzabkar

Thank you so far for all you answers, I hope you can answer some more

1.
The appended bits are created from the file bits
If ALL the file bits was lost / flipped etc., then the appended bits could be used to recreate all the file bits / the whole file?

2.
If one more of the appended bits was flipped, does ECC then correct it / them?

3.
Could only reading a file / data flip bits? (not correct them back, but flip them)
If yes, why would a reading process make "bad" changes to the data?

4.
When using Drive optimization / rearranging data that could flip bits?
I assume yes