Use ram as hard drive cache ?

[johnsmith221]

Hi,


I have a recently built CPU with a 128gb M4 boot/program drive and 1tb WD Caviar green storage drive. I also have 16gb ram (Just because it was cheap)...

Anyway - I was wondering since my storage drive only has 32mb cache and is a green drive it is a bit slow, Would it be possible to use some of my extra ram as hard drive cache ? i have plenty to spare.

My thinking was if i have 1-2gb ram used as hard drive cache it would make my storage drive super fast because it would hardly ever need to read/write and the first 1-2gb of any data transfer would effectively be almost instant...

Any thoughts? is this even possible?

 

[capt_taco]

http://www.romexsoftware.com/en-us/fancy-cache/

 

[Idonno]

This is what I use : http://www.superspeed.com/desktop/supercache.php Here's one of my quotes from another thread:

If you really want to speed things up a little there's a program from a company named SuperSpeed called SuperCache 5 that will help if you have enough ram in your system to dedicate some to it. I dedicated about 8GB out of 24GB ram to it. It's also very easy to turn on or off depending on your needs. It costs $79.95. Not cheep! But, it does help and you can try it for free for 10 days. The Benchmarks on the left are without SuperCache 5 and the ones on the right are with it enabled.

You do want to turn off your Paging File or your Read Access time will suffer slightly.

I think it's also important to note that the Benchmarks I've posted are not representative of real world use. There is a very real performance boost but, it's certainly nothing earth-shattering like the benchmarks seem to suggest. When it comes to the speed/response of your OS the most important benchmark (Read/Acc. time) is the only one that doesn't change at all. That said, It still does make a difference and I think it's worth it. Of course that's me. If you want to see if you think it's worth it or not try the 10 day free trial.

With this program you can assign as many RamCache's to as many hard drives as you want (meaning one cache @ the size of your choosing pr each HD).

 

[InvalidError]

Any thoughts? is this even possible?

Most nix-like OSes have been doing this for 20+ years, Windows started allocating unused RAM to disk cache with XP.

Unless your applications use up your whole 16GB, you should notice that the amount of memory allocated to cache progressively increases as you open and close programs and files until the cache eventually uses up all unused memory.

If you are already using 16GB (hopefully in 2x8GB configuration) with your programs alone, throwing in another 8GB would be cheaper than the fancy-cache software linked above.

 

[photonboy]

Also,
He's talking about the hard drive cache which has little to do with the System RAM.

The speed of the hard drive is limited mainly by its mechanical rotation speed, that's why 5400RPM drives are slower than 7200RPM drives.

Long story short is that Windows does a great job of managing your System Memory and if you want booting and opening of programs to be even faster you should consider an SSD.

 

[InvalidError]

He's talking about the hard drive cache which has little to do with the System RAM.

The OP already has an SSD and specifically asked if it was possible to use system RAM for disk cache, which Windows already does dynamically by default starting with XP.

You cannot beat reloading programs and data off Windows' disk cache with an SSD since the Windows cache can operate at over 10GB/sec while the SSD tops out at ~500MB/s with much higher latency. But for Windows' cache to work optimally, you need enough spare memory for the cache to grow large enough to cover all frequently accessed data on top of all memory used by the applications themselves.

 

[Idonno]

Most nix-like OSes have been doing this for 20+ years, Windows started allocating unused RAM to disk cache with XP.

Unless your applications use up your whole 16GB, you should notice that the amount of memory allocated to cache progressively increases as you open and close programs and files until the cache eventually uses up all unused memory.

If you are already using 16GB (hopefully in 2x8GB configuration) with your programs alone, throwing in another 8GB would be cheaper than the fancy-cache software linked above.

This is a Ram Cache not a Ram Page File and as you could see if you paid any attention at all to the post with "the fancy-cache software linked above" It's a ram cache that works with whatever disk(s) you assign it to, Totally independent of windows as if it were a part of the drive itself. As far as anything being cheaper, All I suggested was that he might want to try it for FREE for a few days to see if he thought it might be worth it to him. I really don't think FREE is to much money.

Here's a tutorial on the Asus ROG forum http://rog.asus.com/forum/showthread.php?6124-How-To-Set-Up-a-RamCache-on-ASUS-X79-Series-Motherboards.

The screen shots on the left are my C drive (2-Sata II SSD's in raid O) with of course all the good stuff windows normally does in a win 7-64bit system.

The screen shots on the right are the same (2-Sata II SSD's in raid O) with an 8gb RamCashe assigned to it. See the difference?

 

[InvalidError]

The screen shots on the right are the same 2-Sata II SSD's in raid O with a 8gb RamCashe. See the differance?

Block-mode device benchmarks have one major flaw: they bypass Windows' built-in file-oriented cache which is how practically 100% of people access their drives after formatting. Device-wise cache software may yield better benchmark scores but not necessarily real-world performance.

 

[Idonno]

Device-wise cache software may yield better benchmark scores but not necessarily real-world performance.

Again I feel that was sufficiently addressed in my first post as well. However there is still a noticeable difference.

Whether it's a worthwhile difference is an individual decision and subjective at that.

Since that is exactly what the OP is asking about, I think it would be worthwhile as well as educational for the OP to try for himself. Since that is free, I don't think it should just be dismissed out of hand regardless of any preconceived notions.

 

[aicom]

Windows has a built-in file cache (as was said above) which programs like CrystalDiskMark purposely circumvent for the purpose of testing. If CrystalDiskMark didn't pass FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH to disable Windows' caching behavior, the benches would look similar to what you posted with the caching program. It might result in slight speed improvements, but remember that there's a reason that Windows doesn't make a massive hard drive cache. RAM is volatile and the larger the cache, the greater the chance of data loss!

 

[Idonno]

My thinking was if i have 1-2gb ram used as hard drive cache it would make my storage drive super fast because it would hardly ever need to read/write and the first 1-2gb of any data transfer would effectively be almost instant...

Any thoughts? is this even possible?

Well I don't know about super fast but, allot faster for sure.

I just did a little test: I put a 8GB iso file on my C drive then transferred it to my D drive which is a 5400 RPM drive just like your WD Caviar green. Transfer time was 50sec without SuperCache (ram cache) enabled for any drives.

I then did the same thing with SuperCache set at 8GB (of ram) for my D drive. The transfer time was 22sec. More than twice as fast.

 

[InvalidError]

If CrystalDiskMark didn't pass FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH to disable Windows' caching behavior, the benches would look similar to what you posted with the caching program.

Glad to see I'm not the only one who figured that out. The Windows file cache numbers should be even better since cache hits end at the file-system layer rather than block-device layer, one less layer of overhead to go through with Windows' cache.

but remember that there's a reason that Windows doesn't make a massive hard drive cache. RAM is volatile and the larger the cache, the greater the chance of data loss!

Windows does create a massive drive cache if there is that much free RAM floating around and you access files to make Windows grow its cache to fill that RAM. Right now, my Windows file cache is at 4.2GB out of 8GB installed with 4.1GB in-use by programs (wonder where the extra 100MB comes from) and would likely grow bigger if I had more RAM installed or fewer programs running.

As far as data loss is concerned, that only happens if your computer crashes/shutdowns while lazy-writes are still in the cache but Windows' cache is periodically flushing lazy writes so the cache is entirely reads most of the time. Losing the read cache is inconsequential since the data can be simply re-read from disk, which is a good thing since it lets Windows grow the read cache as-needed and scrap chunks of it when applications need the RAM for their active working set.

With all the advantages of Windows' file cache (when not circumvented/disabled), there is very little use for aftermarket disk cache applications. Those disk cache utilities look arbitrarily good only because they ignore the benchmarks' no-buffering flags, not because they genuinely outperform the default file cache built into the OS.

 

[Idonno]

It might result in slight speed improvements,

Again I've already addressed that in the first post and while it is subjective the speed improvements (IMHO) are more than slight.

but remember that there's a reason that Windows doesn't make a massive hard drive cache. RAM is volatile and the larger the cache, the greater the chance of data loss!

I've been using "SuperSpeed-SuperCache 5" for well over 1yr now with no data loss what so ever. That's the same thing people were saying about SSD's when I bought mine, no data loss there either even in raid 0 without trim. No slow down either.

"SuperSpeed-SuperCache 5" is so much more than the old fashion Ram Disk dedicated to caching. Just like with any thing new there's always a bunch of people with a bunch of reasons why it won't work. Well technology keeps improving and ideas evolve regardless. That doesn't mean you shouldn't try to understand new technology and it's requirements.

For instance with SuperCache it might not be a bad idea if you turned it off when you are overclocking and still searching for that stable OC. It might not be a bad idea to get a battery back up either, although neither of those is as important as you might think it is. And if your sailing around the world don't forget to bring a parachute for when you get to the edge! LOL

 

[InvalidError]

I then did the same thing with SuperCache set at 8GB (of ram) for my D drive. The transfer time was 22sec. More than twice as fast.

8GB in 22sec would be nearly 400MB/sec... so your source drive must be an SSD and although the copy "finished", the physical writing is going to go on for the balance of the original 50sec due to extremely lazy lazy-write. The copy dialog may have gone away faster but the operation as a whole is nowhere near complete, which can be dangerous.

Also, file caches are mainly meant to accelerated repeated accesses to data so a one-shot file copy is a rather poor benchmark for that.

 

[Idonno]

8GB in 22sec would be nearly 400MB/sec... so your source drive must be an SSD and although the copy "finished", the physical writing is going to go on for the balance of the original 50sec due to extremely lazy lazy-write. The copy dialog may have gone away faster but the operation as a whole is nowhere near complete, which can be dangerous.

Well, Duh and I don't believe I was shy about saying my C drive was an SSD just like the OP and the drive I was transferring to was a 5400RPM HD again just like the OP. As far as danger what's the worst that can happen? Even if the PC were to shut down unexpectedly I'd just transfer it again. If your using lazy-write you wouldn't delete the source until lazy write is done with it but that goes back to my earlier post were I said "That doesn't mean you shouldn't try to understand new technology and it's requirements." You really need to actually read the posts.

Also, file caches are mainly meant to accelerated repeated accesses to data so a one-shot file copy is a rather poor benchmark for that.

Actually it's a very good bench mark since that's where the OP will most likely store his larger files and that is in fact the HD (the 5400RPM HD) he was interested in speeding up. Was it a complete overview of the benefits? Of course not. I use SuperCache on my C drive, that's where my OS and programs are and yes SuperCache Improves on all aspects over what windows by itself does. But then again, I'm the one that's been using it for over one year not you.

And One final thought, I quite clearly said in my 1st post:

"I think it's also important to note that the Benchmarks I've posted are not representative of real world use. There is a very real performance boost but, it's certainly nothing earth-shattering like the benchmarks seem to suggest. When it comes to the speed/response of your OS the most important benchmark (Read/Acc. time) is the only one that doesn't change at all. That said, It still does make a difference and I think it's worth it. Of course that's me. If you want to see if you think it's worth it or not try the 10 day free trial."

Not all that other garbage you both seem to be insinuating. But then again you would have to actually READ the posts to know that.

 

[InvalidError]

As far as danger what's the worst that can happen?

Modified file system entries getting trashed and taking out chunks of the file system on the drive. Quite common with people who carelessly disconnect USB drives while in a hurry.

"That doesn't mean you shouldn't try to understand new technology and it's requirements."

There is nothing new about disk/file caching, it has been around since DOS 3.3 / Windows 3.x, even earlier from 3rd-party developers and other OSes.

The only significant differences performance-wise between Windows' built-in file cache and aftermarket caches is that Windows' does not allow as much lazy-writing and it honors no-buffering flags from software.

Actually it's a very good bench mark since that's where the OP will most likely store his larger files and that is in fact the HD (the 5400RPM HD) he was interested in speeding up.

Except the operation is not really "sped up" since full completion including flushing lazy writes will take just as long as the regular copy without cache, it is only an illusion of speed-up, you just end up wasting an equivalent amount of time copy-sitting until lazy-write is done to know when to delete the originals.

If the OP wants something to help his slow drive get writes done (as in DONE) faster, there is no miracle. Neither is there one for one-time reads.

 

[Idonno]

Oh, stop. You don't even have a clue. The OP asked an honest question and you have just now taken it to ridiculous proportions. You really should be ashamed of yourself. I could trash every dumb thing you just said but in the interest of sanity I'm just going to leave it alone. If the OP's interested he can follow the link I posted and learn what some of the fastest machines in the industry are doing. If not that's OK too.

 

[johnsmith221]

Best answer selected by johnsmith221.

 

[VictorLimaMike]

Here's a tutorial on the Asus ROG forum http://rog.asus.com/forum/showthread.php?6124-How-To-Set-Up-a-RamCache-on-ASUS-X79-Series-Motherboards.

I built my first gaming rig this past June and have RST setup and am very happy beyond words with the final setup. But, I also still have my fairly new "big box store" HP Pavilion.

It isn't the quickest obviously and without going all out on a 2nd HDD for raid config, CPU, RAM, etc; would the RamCaching software provide any significant boost in performance?

$30-$70 bucks for software versus buying new hardware seems to be the more practical approach. But I'm new to this stuff so begging your pardon in advance.

I expect to keep this HP for a while because it has all the home office stuff, etc. but still want to future proof it to some extent.

Reading your post on ROG seemed to lean more towards a custom built rig.

Thanks for your hard work.

 

[Idonno]

Well, I guess that depends on what you consider a "significant boost" My best advise would be to try it and judge for yourself since "SuperCache 5" has a free 14 day trial.

As far as my opinion as to whether it will offer a significant boost to a mechanical hard drive: Yes absolutely! While I do notice a difference using SuperCache with my SSD's, I can also understand how most might not find the difference a "significant boost".

However on my old XP (32 bit) PC with only mechanical hard drives the difference is much more pronounced and can only be described as "significant".

On that old XP PC programs used to open very slowly, now they open 5-10X faster. I used to joke that when you opened Firefox you could go to the bathroom, get a cup of coffee, smoke a cigarette and still make it back just in time to see Google appear. Now by the time you say one-Mississippi it's there and that of course is just one example of many.

Another nice feature of SuperCache is it can make use of any unused ram. For example with 4GB ram my 32 bit XP PC only sees 3.25GB. With a 1GB RamCashe the .75GB that the OS doesn't see is used so I still have 3GB left for the OS.

 

[VictorLimaMike]

Well, I guess that depends on what you consider a "significant boost" My best advise would be to try it and judge for yourself since "SuperCache 5" has a free 14 day trial.

As far as my opinion as to whether it will offer a significant boost to a mechanical hard drive: Yes absolutely! While I do notice a difference using SuperCache with my SSD's, I can also understand how most might not find the difference a "significant boost".

However on my old XP (32 bit) PC with only mechanical hard drives the difference is much more pronounced and can only be described as "significant".

On that old XP PC programs used to open very slowly, now they open 5-10X faster. I used to joke that when you opened Firefox you could go to the bathroom, get a cup of coffee, smoke a cigarette and still make it back just in time to see Google appear. Now by the time you say one-Mississippi it's there and that of course is just one example of many.

Another nice feature of SuperCache is it can make use of any unused ram. For example with 4GB ram my 32 bit XP PC only sees 3.25GB. With a 1GB RamCashe the .75GB that the OS doesn't see is used so I still have 3GB left for the OS.

Awesome, thanks. Since it is the home office PC, there are certain things like opening apps and multitasking that I put it through on a regular basis. I'll give it a try.

 

[TobeyZ]

I use SuperCache on my C drive, that's where my OS and programs are and yes SuperCache Improves on all aspects over what windows by itself does.

I just looked at the SuperSpeed site and it states:
"SuperCache is not recommended for a boot volume ( usually C: ) .
Use only on non-boot data volumes."

As my main reason to consider SuperCache would have been to write to RAM cache versus my SSD where my OS resides in order to extend the life of SSD and maybe also get a slight performance gain. Apparently, this is not recommended..

 

[Idonno]

Yea, It's true it does say that. I believe this is mostly due to the fact that a loss of power could cause some data loss especially when using lazy write. If you use a battery back up this concern is mostly eliminated.

I for example have been using SuperCache on the boot volumes of the PC in my signature as well as three more of my six PCs for over a year now with only positive results. I use lazy write on all four @ anywhere from 5sec to infinite.

I have had only positive results. The way I see it is, like raid it does add risk but, I feel the benefits out way the risks with both raid and SuperCache on a boot volume. But that's me and like I said SuperCache hasn't caused any issues for me, just benefits.

 

[TobeyZ]

Thx 4 the reply Idonno...

Your reply makes sense. SuperSpeed would naturally want to strongly advise against using any process that may result in important data loss.
As I use battery backup, and given your track record I have decided to implement this on one of my systems to evaluate for awhile.

 

[Idonno]

Thx 4 the reply Idonno...

Your reply makes sense. SuperSpeed would naturally want to strongly advise against using any process that may result in important data loss.
As I use battery backup, and given your track record I have decided to implement this on one of my systems to evaluate for awhile.

No problem "TobeyZ", Thats a good choice. If you don't think it's worth it for you it won't hurt your wallet anyway.