Freeing Up Capacity On An SSD With NTFS Compression

[ProDigit10]

[citation][nom]acku[/nom]Correctomundo. Compression involves replace repeated occurrences of data with references to a single copy of that data existing earlier in the input (uncompressed) data stream. That's why it's not right to think of a compressed archive as a container that stores any given file into a discrete space. If anything, the files kind of overlap in a big mixing pot.When you compress on the fly, you have to completely decompress all the files in an archive and recompress it when you're done. Hence it's all random transfers for the most part.It's not a sequential transfer. Plus it's already precompressed data.[/citation]
Then they should not call it 'on the fly' compression!
On the fly indicates compression done in RAM, BEFORE written to the disk in a permanent way.

 

[clonazepam]

[citation][nom]wolfram23[/nom]...and Code Master games (F1 2010, Dirt 3) like to store 1gb replays in there)...[/citation]

Thanks for that. I went to C:\ProgramData\Codemasters... and between Dirt2 and Dirt3 replay videos, I cleared up another 4GB!

The biggest offender is Bioware so far that I've seen. Between Dragon Age, Dragon Age II, and Mass Effect II, all of the DLC is in \My Documents\Bioware\... and is taking up a whopping 5GB. I understand putting save files neatly in a user's documents area, but the DLC? I knew it was there from the beginning, but this seemed like as good a place as any to complain about it. I guess I should get around to finishing these titles so I can clean it up.

 

[funguseater]

Yeah!!! Good old Stacker, brings back all kinds of MFM memories...

 

[pcito]

Probably a good idea to compress only the read-only or mostly read-from folders, and not compress the folders that are commonly written to. It seems like this would give you most of the advantages while minimizing the write-cycle problem. Correct me if I'm wrong.

 

[dalauder]

[citation][nom]ProDigit10[/nom]I have UD since 2008, and their claim is justified. They base their thought on the surface speed on the outter rings of a disk is faster than the inner speeds.It would be the same if the engine would be able to rotate the platter from 7200rpm to 10.000rpm on the inner disk, and slow down on the outter, but since hardware manufacturers want to keep cost low, they equip their drives with a single speed motor.[/citation]Yeah, but a variable motor is typically less reliable.

 

[dalauder]

[citation][nom]clonazepam[/nom]Thanks for that. I went to C:\ProgramData\Codemasters... and between Dirt2 and Dirt3 replay videos, I cleared up another 4GB! The biggest offender is Bioware so far that I've seen. Between Dragon Age, Dragon Age II, and Mass Effect II, all of the DLC is in \My Documents\Bioware\... and is taking up a whopping 5GB. I understand putting save files neatly in a user's documents area, but the DLC? I knew it was there from the beginning, but this seemed like as good a place as any to complain about it. I guess I should get around to finishing these titles so I can clean it up.[/citation]I actually keep My Documents on my storage array. I don't find the performance accessing pictures to matter nearly as much as keeping the space available for programs on my faster RAID 0 array.

 

[jowunger]

I wonder how much speed would be sacrificed/speedup if put the Virtual Memory on a HDD / on a SDD.

 

[rantoc]

I would personally compress programs and the os but not "stream-readed" games like fallout3, new vegas, skyrim ect that loads during game play due to the additional decompression needed.

Cheers for an an intressting review.

 

[Guest]

how come this increases writes in nand, but in sandforce it decreases write amplification to values below 1 ?

 

[Krusher]

~1 year ago I ended up using compression on some Steam files to free up some space for more games. If you have a quad-core or better, you might try compressing *.vpk, *.gcf, and *.bsp files. The article confirms here that it is marginally slower, but I haven't noticed any difference myself in the games I've compressed. SSD's are so fast to begin with.

 

[PhilFrisbie]

I use NTFS compression in some virtual machines to keep the virtual drive size down.

 

[Traciatim]

[citation][nom]ProDigit10[/nom]I have UD since 2008, and their claim is justified. They base their thought on the surface speed on the outter rings of a disk is faster than the inner speeds.It would be the same if the engine would be able to rotate the platter from 7200rpm to 10.000rpm on the inner disk, and slow down on the outter, but since hardware manufacturers want to keep cost low, they equip their drives with a single speed motor.It only works for harddrives, not for ssd's, and speed could go up from 18MB/s read speeds on the inner rings, to 80MB/s on the outter.There have been many reviews of HDTACH etc done on harddrives on TH, proving this.[/citation]

I've been using UD for over a year and know how much I notice the performance difference, my main question was if enabling compression on EXE and DLLs across your drive actually did end up helping or hindering performance.

For everyone with a spindle drive, I'd recommend taking a look at Ultimate Defrag. It rocks.

 

[danwat1234]

[citation][nom]Chesteracorgi[/nom]Do you have any data on how much using compression shortens the life cycle of the SSD? It would be most helpful as those with smaller SSDs are the likely candidates for using compression.[/citation]
Don't worry about it. Even small MLC SSDs can handle hundreds of Terabytes of data written to them during their lifetime. http://www.xtremesystems.org/forums/showthread.php?271063-SSD-Write-Endurance-25nm-Vs-34nm

It's not an issue.
I use NTFS filesystem compression on my G2, got over 10GB more space, very happy. NTFS compression is not CPU heavy at all. Unless you are running an Intel Atom, don't worry about it.

On my storage drive I often use NTFS compression on my bittorrent folder so it will sparse all the files that aren't 'filled' yet. Fragmentation becomes huge however, at one point on my 2TB drive, it had over 4 million file fragments! Defragging once all my downloads were done brought that number way down.

 

[joe_newbuilder]

As a point of reference, after reading this article I enabled ntfs compression on user folder in my 60gb boot drive. I had 6 gb free on it. After compression I had 6.1 gb free. Most of the files that took up a bulk of the space were already compressed formats. The gain is so small I'd say it wasn't worth it.

It's going to be very dependent on the type of files taking up space on your drive.

 

[ilysaml]

We still need an article about Raid0 & TRIM.

 

[eddieroolz]

Interseting. I've always held to the notion that NTFS compression wasn't very effective at saving space while causing performance penalties. While the latter may have been because of laptop systems, it does seem that the space savings are quite worthwhile only just 70GB of test data.

 

[Guest]

Another way to save space in an SSD is to re-direct all of your user profile folders to an HDD. It saves me quite a bit of room to redirect things like 'Downloads' to the HDD and it doesn't really slow my system noticeably. I suppose there is a hit to performance when unpacking an archive from the HDD, but, I'm mostly worried about boot times and game performance. I keep all of those files on my SSD.

 

[Guest]

If ntfs compression algorithms are similar to deduplication algorithms, you would essentially do less writes of the same block patterns over and over. Also, it would have been nice to see the sata HDD comparison averages for compressed and non compressed with the same SSD data.

 

[duxducis]

ok just did test on my Adata 60 gig ssd, compressed it and properties say size on disk is less saving me about 5gig, but when i go whole drive properties , i have lost space free space was 25gig after compressing users and windows folders free space dropped to 18gig 7 gig just vanished any ideas why????

 

[JimmiG]

It's probably best to selectively apply compression where it really makes a difference, rather than compressing the whole drive. For the most benefit, you should apply it to highly compressible data that you rarely modify.

Experiment a bit to find out which folders or files compress the most. You can copy the folders to a regular hard drive so you don't wear out the SSD while testing.

 

[Guest]

Gave this ago on my Cosair Force 3 120GB (system with 8GB ram and high end Quad CPU). Free space before compression 33.9 GB, free space after compression 38.2 GB. After compression ran and a restart the first boot up was like a mechanical drive, the icons even popped up on desktop one at a time, HDD light was stuck on for ages with slow response from any apps I tried to load.

Uncompressed drive and everything back to its normal fast load times. Would not recommend this to anyone not fighting for every last GB from their drive. Also I recovered more space from moving swap file and temporary internet files to a mechanical secondary drive with no performance impact.

 

[tigerwild]

IMO, you should all do as I am, skip the NAND flash and go STRAIGHT RAM. I use 80GBytes with my swap files and temp all mapped to a 60GB RAM disk. At 52,000 iops and 2,500+ MBps burst, 700+ MBps sustained very little can compare. Plus I don't have to worry about wear levelling.

see the ANS-9010 RAM drive (yes it was expensive, but OOOOH so worth it, as it gets FASTER with the more advanced chipset you connect it to!)

In reply to below= I believe the OS writes the data uncompressed, then in a second thread goes and compresses it in the background replacing references to the data as it is compressed (thus you get nearly double the writes)
-----------------------------------------------------------------------
I've been wondering about this very topic for a while now.


However, in the conclusion, it is stated that compression ends up writing more vs. uncompressed NTFS, thus consuming more PE cycles. Shouldn't the opposite be true? When writing to the file system, if a file is compressible it should take up less space and therefore conserve more PEs (though actually compressing the files for the first time should result in more writes).

Why does on-the-fly compression result in more writes even though the amount to be written is smaller?

 

[blazorthon]

NTFS compression seems like it would do better on a non-sandforce drive.

 

[mutex7]

You stated "wear leveling algorithms don’t work efficiently when the drives are completely filled".

So, this begs the question...how many gigabytes (or what percentage of an SSD) should be left free to allow wear algorithms to work efficiently? Thanks.

 

[nurgletheunclean]

I have been using NTFS compression with my intel and crucial based SSD and think it's been good. However NTFS compression is in my opinion very antiquated especially considereing there are many both faster and efficient code out there. FreeARC comes to mind with faster than wire speed compression at zip levels. QuikLZ is another super fast API. I am guessing that these are the algorithms used by sandforce controllers. I would really like to see Microsoft update the ancient NTFS compression. Furthermore as someone stated earlier NTFS compression doesn't compress in memory and then write to disk so you aren't saving yourself any write endurance.

I remember back in dos/win3.1 days a product called stacker, which created a compressed volume in dos. I feel there's a big opportunity for a company like that to introduce a new file system compression layer.

FreeARC also compresses in memory and then writes to the disk, greatly improving write endurance.