Question Newly downloaded files on external hard drives are slow to copy/move

[azuree125]

Facing this weird issue where newly downloaded files videos (tv shows, movies) are are slower to copy/move compared files/videos already present on the drive.

I have two 2.5 inch portable HDD's (one seagate onetouch 4tb and another WD passport 5tb) connected to my pc where I keep all my tv shows/movies downloaded. But from the last week or so I'm facing this problem where newly downloaded files are much slower to copy to let say my PC's internal HDD.

Now when I tried to copy those tv shows/videos to another drive the speed would barely touch 15MBps while other similar videos previously downloaded on both drives get copied at upwards of 70MBps.

Also if I make a copy of the video (the one that is slow to copy) on the same external drive (both passport and onetouch) then try to copy/move this “copy” to internal drive this now goes at full speed. I’ve tried drives on other PC’s and got the same behavior there too. And for two separate drives to develop the same issue at the same time is a bit too much of coincidence. I feel there’s something else at play.

Please any help is appreciated.

 

[dwd999]

Since you're talking about HDDs perhaps they need to be defragmented so that new material can be copied to one contiguous space.

 

[azuree125]

Both drives have over one 1TB free space and I've used these drives with as little as 20GB free but never had any such issue. Also this does not explain how after making a copy of the newer files, the "copy" transfers at full speed. I'm honestly stumped at this point.

 

[LinuxDevice]

Maybe this is related, not sure.

Whenever you have a file to move around on a single disk, only the metadata changes and no actual "storage" changes. When you copy from one disk to another, then all of this has to be copied, and this is no longer a simple metadata edit.

Any disk which is present on a typical Windows/Mac/Linux system probably has some internal cache enabled. That cache may not be very big compared to the file size, and tends to help with random access, but is limited. No cache or a cache miss means no advantage from this.

There is also "buffer", and this is much bigger than cache, and is likely what you are experiencing. Buffer is not part of the disk. Most operating systems will use RAM which is not in use as a buffer (if the system needs more RAM, then the buffer is silently discarded and used for whatever else needs it). In order for a buffer to apply the content has to have been accessed once. This buffer will be applied to on the source disk and a separate buffer on the destination disk. If you access that particular disk a second time, then the buffer is read instead of the actual disk. This is a dramatic increase in performance.

If you have never copied (in that boot) a large file from external to internal storage, then you don't have a buffer, but the second time it is read, the buffer will cause the read to be much faster; if you edit that file, then it might slow down again because the buffer has changed and would need to updated.

It really sounds like you are comparing a first operation (with no buffer) to a second operation (which would use buffer and be much faster).

I'll add that if your system does not have enough RAM, then you'll lose buffer. If the RAM is used at the start, then you won't get much buffer. If the RAM is used after you've read from that disk, then the buffer you had will go away until RAM becomes available and you have read from the disk one more time.

Someone else mentioned fragmentation, and if this is an old tech drive with mechanical spinning platters, then that is quite possibly part of the issue. Solid state drives don't really have that issue (and defragmenting a solid state drive would reduce its life without improving anything). A defragmented drive stores content of one file continuously, and once data seek begins, more seek is not generally required; it would just read from start of file to end of file. If the file is fragmented, then it has to seek pieces of the file all over the drive, and seek time is much slower than reading when the head assembly is already where it needs to be. Solid state drives simply have addresses and no physical platter.

Related to what I said about defragmenting a solid state drive lowering its life, a solid state drive has a lot of "cells" for data storage. There is a table of cell addresses which have the information on which address to find a given file (or fragment) at. Every time a cell is written its life decreases, and eventually a cell will fail. This is why writing or editing has hardware and firmware inside of the solid state drive which spreads out which cell is used; each edit will go to a new (unused) cell rather than changing content in one cell, ad this keeps a given cell from wearing out. This is "drive/wear leveling". If a drive is too close to capacity, then wear leveling can slow down by quite a bit. A near-capacity solid state drive is slower than one with some unused reserve. Incidentally, a buffer on top of a solid state drive also increases the life of the drive; changes are accumulated in the buffer prior to flushing to disk, and so there are fewer changes to the actual disk.

 

[azuree125]

Both drives are less than 4 months old. Infact I also tested another 1TB portable WD HDD I had and it showed the exact same problem - newer downloaded files are slower to move and copy compared to the ones already stored on the drive.