Strange data corruption with large files

[freelunch]

I have 4 machines.
1 windows 2000 server with sata-ii drives
1 xp laptop sp2
1 xp laptop sp3
1 vista laptop sp1
All avg pro v8

When I copy a 10gb file from any machine to any other machine(copy/xcopy) and follow this with a "fc /b" all copies are identical.

However.... if I use ftp, rsync, or any other delta copying method I always get errors from "fc /b", usually 2 bits (02 -> 00) sometimes a few more.

When I overwrite the same file with copy /b or xcopy or even xxcopy, no problems, 100% ok.

Since this is happening to 4 different OS's and hardware systems I am excluding bad HD's and ram.

I have tried this with a smaller file like 2gb, no problems here.

I've been running tests for days now, recompiling rsync and other delta tools, added extra layers that run a md5 and sha1 checksum (who always report that the data written to disk matches both checksums) and still fc detects at least 2 wrong bits. xxcopy again and all is well.. slightly going insane here...

Any ideas?

 

[freelunch]

Nobody any ideas? could someone do a test?
Ie. take a DVD and copy this as a ISO to PC-1
take another DVD and copy this as a ISO to PC-2
Install something like deltacopy and (r)sync one ISO with the other between pc's.
Then run a fc /b between pc's ISO's.

 

[Flakes]

if the file still works then the extra two bits are probably just header information that the ftp program has put there.... don't worry about it.

 

[freelunch]

I wish... the bits are located at offsets like ecd09981 ie. anywhere beyond the 2.5g range. and no its not a 16/24bit limit problem.

 

[freelunch]

Just a quick twitter, I now suspect windows folder compression, its the only thing commonly used between pc's.

 

[freelunch]

Yep, confirmed, its windows disk compression doing this on all platforms, so not just a bug in home server, compress files larger then 1gb and you risk corruption... it took a few days to test this with 400gb, decompressing and repeat.