Files transferred to new 8TB external drive are larger than originals

[dam151]

This a Windows 10 computer. The external HDD is exFAT as formatted from manufacturer.
Files transferred to the 8TB external drive are larger than originals. One single file is 28.3MB (29,709,239 bytes) but on the 8TB drive it has a "size on disk" of 29.0MB (30,408,704 bytes). If only 1 file, maybe not such a big deal but 83.8 GB (90,026,903,391 bytes) becomes 106GB (114,715,262,976 bytes) "size on disk". I'm guessing this is a default base size based on 8TB volume but at this rate my 8TB will turn into 6. What can I do about this? Partitioning?

Also, I know the named drive size is never the actual drive size but the 8TB is reporting 7.27TB capacity which seems low to me (should be 7.4?). The Drive Properties reports Used space at 143GB but in addition to the folder referenced above, there are just 2 folders of combined size <1GB and an ISO file just under 4GB. How do those plus the 106GB = 143GB?

What can I do to maximize the usable space on this hard drive? Reformat? Partition? Thanks!!

 

[seagate_surfer]

Hello dam151, if you have a Seagate drive, suggestion would be downloading DiscWizard and use Extend Capacity Manager (section 6.3) to see if that solves it. If not formatting would be required. Best of luck.

 

[Solandri]

This a Windows 10 computer. The external HDD is exFAT as formatted from manufacturer.
Files transferred to the 8TB external drive are larger than originals. One single file is 28.3MB (29,709,239 bytes) but on the 8TB drive it has a "size on disk" of 29.0MB (30,408,704 bytes). If only 1 file, maybe not such a big deal but 83.8 GB (90,026,903,391 bytes) becomes 106GB (114,715,262,976 bytes) "size on disk". I'm guessing this is a default base size based on 8TB volume but at this rate my 8TB will turn into 6. What can I do about this? Partitioning?

Default custer size for exFAT on terabyte volumes is 128 kB. By comparison, the default cluster size for an 8 TB NTFS volume would be 4 kB. A cluster is the smallest unit of data that can be stored on the partition.

https://support.microsoft.com/en-us/help/140365/default-cluster-size-for-ntfs-fat-and-exfat

Basically, this means:

• 
  ■Any file smaller than 128 kB takes up 128 kB of space on your external HDD. The extra space is padded with zeroes. So the amount of wasted space due to files smaller than 128 kB is (number of small files)*(128 kB) - (actual size of data in those files).
  ■Files larger than 128 kB have their storage space rounded up to the smallest integer multiple of 128 kB (the extra space is padded with zeroes). The amount of wasted space is then on average (number of large files)*(64 kB) since on average half of the last cluster is padded with zeroes.

Switching to NTFS with 4k clusters would reduce the (128 kB) in the above calculations to (4 kB). If you've got a lot of small files, this could be a substantial space savings.

exFAT was designed specifically to allow external storage devices to be used between Windows and OS X. If you don't plan to use this drive to share files with Macs, then there's no reason to keep the drive formatted as exFAT. Reformat it as NTFS to reduce the cluster size to 4 kB. If you do need an exFAT partition to share with Macs, then you may want to try creating a 32GB exFAT partition (with 32 kB clusters) and format the rest as NTFS. Copy data which you plan to share to/from the exFAT partition only as needed. (You can also try formatting the drive as exFAT with the smaller 32kB clusters, but that tends to balloon the amount of space wasted due to filesystem overhead. By how much exactly is unknown until you try it, but the default cluster sizes were chosen because they minimized space wasted to both filesystem overhead and padding clusters with zeroes in average use cases.)

And 8 trillion / (1024^4) = 7.27. So yes, 7.27 TB is correct. 8 billion bytes would be 7.45 GB.

If you encounter a similar problem even with NTFS (i.e. you have a lot of files which are much smaller than 4 kB), then the usual solution is to pack the folder(s) containing those small files into a zip archive. That will consolidate them into a single file so they don't take up as much space on the drive. And Windows 7 and newer can navigate zip archives natively (as if they were just folders). Actually you could do this on exFAT too if the files are read-only (e.g. lots of small PDFs). It usually doesn't work for files which need to be written or modified.