RAMCloud: The Idea of Storing All Data in RAM

Page 2 - Seeking answers? Join the Tom's Hardware community: where nearly two million members share solutions and discuss the latest tech.
Toggle sidebar Toggle sidebar
Status
Not open for further replies.
[citation][nom]ojas[/nom]I thought SSDs use non-volatile NAND RAM? or is that NAND flash? :\Why do they call it a "could" system? Isn't this just a ram-based server? Cloud is like, for storing data and things right? "Run your shitz of the cloud". I can't see the point of it for cloud storage, would be too expensive. What's the point of the super-low latency if your ping to the RamCloud will be 200ms? A movie streams a ms late, big deal...If it's used for regular server hosting (like game servers, etc) then i can see the point, because computation time would reduce.[/citation]
NAND RAM is much slower than DRAM and can wear out after certain number of writes and erase cycles.

Yes someone's ping could be 200 due to internet connection, but usage of RAM for storage could allow that 100-1000X people have 200ms ping where HDD/SSD allows for X people to have.
 
[citation][nom]ThisIsMe[/nom]Not sure if the researchers have taken into consideration the added cost from having to use only buffered ECC RAM. The unbuffered non-ECC RAM some of the commenters here are referring to is considered volatile for more than just what happens when the power turns off. After some time the data stored in this RAM becomes corrupted and would no longer be the same data. Ever wonder why some programs after being open and running for several days can sometimes start to act funny after a while?For the past 20 years or so OS's have used redundancy to help with this issue via the swap file on the HDD. Using buffered ECC RAM can eliminate about 99% of this problem, but can cost 2-3 times as much in some cases. Although, I suppose you could offset the cost by using twice as much "regular" RAM and using half for redundancy. Still, the best option would be to store the data on a typical storage device and have a newer better buffering system which keeps and accurate and updated copy in the RAM "cloud" which could keep the data integrity in check. This way there would only be the older latency issue initially when the data is being uploaded or altered, or when it needs to be rebuffered because of corruption. Plus that thing about the power going out would be a non issue.[/citation]
Or you can, as researches did, use regular HDDs for backup and redundancy.
 
[citation][nom]ojas[/nom]BTW does anyone else notice a CPU usage spike when you scroll from the article to the comments? Happens somewhere at the end of the article...been noticing this for a while.Use chrome, 32-bit, 3.2GB RAM and a Core 2 Quad. Tried on a C2D laptop and happens there too, had used chrome again.EDIT: Just tried with IE, spike happens but you cant notice it. Chrome actually stutters a bit...[/citation]

Tested it out.. you're right.
 
[citation][nom]Netherscourge[/nom]Sounds awesome - until the power goes out.;([/citation]
You are 100% correct. No UPS lasts forever. Even with generators all it takes is a single datacenter to have an outage just a little greater than their backup power. Just ask RIM.

I'ts a horrible Idea if you not constantly replicating to disks, wich will probably not keep up.
 
Purely in-memory datastores, like memcached and redis, have been around for awhile. MemcacheDB and RedisCluster (still in progress) are projects which manage large clusters running these services. Redis and memcached were both benchmarked handling in excess of 100k reads/writes per second on consumer grade laptops a few years ago. Redis can even snapshot to disk fairly frequently so that it isn't as volatile.

It's great to see more people talking about this as a viable technology, but these researchers are a few years behind the times. The algorithms for making large clusters handle failing nodes are in wide use for disk based storage systems, the software for memory-only storage services are in common use, combinations there of, as mentioned above, are already being built and have been talked about for years.
 
[citation][nom]ojas[/nom]BTW does anyone else notice a CPU usage spike when you scroll from the article to the comments? Happens somewhere at the end of the article...been noticing this for a while.Use chrome, 32-bit, 3.2GB RAM and a Core 2 Quad. Tried on a C2D laptop and happens there too, had used chrome again.EDIT: Just tried with IE, spike happens but you cant notice it. Chrome actually stutters a bit...[/citation]
Nope. Running the latest version of Chrome, an i5 750, 8GB RAM, Win7 64-bit, and a geforce 550ti. I looked while scrolling down and the CPU usage was only at 3-5%.
 
Gawd, this is 'tarded. Smart server applications cache everything they need in memory anyways this is like something a 14 y/o would think of, I'd hate to think of the "research" that went into this idea.
 
My Clan is using Ram Disks already for our Minecraft Server. World Saves and Map generations are quick. Though I have no problems already with current cloud services.
 
This sounds great I wonder why it's never been done before .... oh wait, it has. Flashback to the RamDisks of the 90s (sans LIM card).
 
Yeah somone correct me if im wrong, but i beleive you can already partion some of your ram as a useable drive. I'm not 100% on the specifics (ie does it simply load pre defined programs into an alloted space or does it automatically load the things causing most dely intell SSD boost style) but honestly, realising the throughput of ram is higher than that of SSD's is all theyve done here. WOW. Go Stanford. . . . .
As for volatility issue the obvious is to have an SSD storing the data when powers off and transfer that data to the RAM drive on boot.
So yeah. Not impressed.
 
ram cloud! me like.
why not use something like capacitors in the ram modules or high speed flash memory to store data in case of outage and use ssd's to cache the data and then store it on a disk drive as back up. only problem(latency/lag) with that would be to bring the data back from hdd to the ram. from fastest(main) to slowest(storage/backup): ram > flash cache > ssd cache > hdd(storage/backup).
non volatile ram would be awesome too.
 
[citation][nom]livebriand[/nom]Nope. Running the latest version of Chrome, an i5 750, 8GB RAM, Win7 64-bit, and a geforce 550ti. I looked while scrolling down and the CPU usage was only at 3-5%.[/citation]

Not saying it's a chrome prob, i think it's a web page prob.

And your PC is pretty good, maybe it can take it? I dunno...try scrolling up and down across the entire page repeatedly, very fast.
 
used to setup a Ramdrive back in the old DOS days to speed things up by putting the temp dir and swap file there. i assume windows is doing this automatically in the background with the system cache. and hard drives have their buffer caches too. databases use a lot of ram to as data buffers too. but sometimes i wonder if there is more we could do to manage all the ram we have on most new systems.
 


Well, I thought that was normal for most of the pages with heavy content, like facebook, and even hotmail (but I think that one has real problems),
My processor is a Phenom II X4 running in XP limited to 4GB RAM (limited to 3.3) in dual channel at 1600 cas 7 , and I experience this stuttering while using CnQ almost in any page, but when disabled or using Phenom MSR Tweaker or K10Stat for custom CnQ, it disappear.

Now my chrome version is an old developer 12.0.712.0 unknown (with 120% zoom, at 1080p, with a lot of extensions including Smoothscroll, and AdBlock), my processor is overclocked to 3.7 but it just jump to 2.7 when scrolling TH pages with a usage of no more than 20%.

The page have experienced some changes recently so maybe that is why has become a little heavier, but still think is maybe due to your processor.
 
They could also title this article "The powers that be REALLY REALLY want to get your data away from you!" ...so lets create this near sounding scenario that cures all the ills of storage on your PC.

SSD has latency problems? . LOL
 
Why this would be creepily awesome for the consumer, performance-wise, the cost of a 128GB RAM drive would seemingly not be worth even the spectacular performance. I want one.
 
In contrast to many of the above posts i think the article was more about the size of the RAM drive then that they were using a RAM drive as a RAM drive is nothing new but a 64TB RAM drive is. As stated above many people already use RAM drives for their severs and thats good and all just they made a storage sever RAM drive which i hope they are keeping backed up as a single power failure to it would be a massive loss of information.
 
Not exactly new news. :) I remember being able to store a small Mac OS install in memory on an old Powerbook. I also remember RAMdisks for DOS. I seem to recall one that would let you take any memory DOS couldn't address at the time and use it for blazing fast program storage.
 
[citation][nom]Krnt[/nom]Well, I thought that was normal for most of the pages with heavy content, like facebook, and even hotmail (but I think that one has real problems),My processor is a Phenom II X4 running in XP limited to 4GB RAM (limited to 3.3) in dual channel at 1600 cas 7 , and I experience this stuttering while using CnQ almost in any page, but when disabled or using Phenom MSR Tweaker or K10Stat for custom CnQ, it disappear.Now my chrome version is an old developer 12.0.712.0 unknown (with 120% zoom, at 1080p, with a lot of extensions including Smoothscroll, and AdBlock), my processor is overclocked to 3.7 but it just jump to 2.7 when scrolling TH pages with a usage of no more than 20%.The page have experienced some changes recently so maybe that is why has become a little heavier, but still think is maybe due to your processor.[/citation]

well, i've the latest version of chrome, 1024x768, ABP running too....and as i said, usage spikes happen with IE too, but there's no stuttering. It's unlikely to be a browser issue.

This has been happening for quite some time, before the changes. Yes, usage hits 22% when i rapidly scroll up and down, but that shouldn't happen i suppose? Compared to Wikipedia, i dont see any spike till i start scrolling rapidly and repeatedly, and even then the usage varies from 8% to 20% at the most. On chrome. Plus my mouse contributes a bit to the usage as well. (800 dpi, USB, i use Logitech's SetPoint). But that still doesn't account for that spike during normal scrolling, as i use the side buttons on the mouse.

May be my proc, or maybe CnQ and SpeedStep... But it doesn't make sense, why does it happen just at a particular point? There's something up with the code after the article ends, imo.

I was hoping that someone from the site could check it out, that's all...which is why i put it up here, wanted to confirm that it happens at all...

BTW, WHAT jumps from 3.7 to 2.7? Clock speeds? Shouldn't it be the other way around? i'm unable to follow you there...it's overclocked, but then it underclocks itself on TH pages, and CPU usage jumps to 20% (which is the problem)?
 


Hmm... I tried to notice the stuttering in some heavier pages (newegg, modDB, devianart, TechpowerUP) with no success, then I returned to TH and now I notice it, maybe because now I opened a newer article that wasn't in the cache, I'm not sure why I didn't notice it before if it was present (maybe because I am a little accustomed to some slower machines lately), it happens just before showing the comments section (maybe the comments are loaded while scrolling).

Also about the processor clock thing I was talking before, It idles at 800 MHz, jumps to 2.2 then to 2.7 and then to 3.7 depending on the usage, and for scrolling it just need to jump to 2.7 with a usage of 20% in that current state, about that, maybe I was't able to notice the stuttering because the CPU needs like 8 seconds to return to 800 from 3.7 GHz and if I make the test when the CPU is not in its lowest C-state the issue will not be noticeable, and maybe that is why in some Sandy CPUs it will not be noticeable.

 
[citation][nom]ojas[/nom]Not saying it's a chrome prob, i think it's a web page prob.And your PC is pretty good, maybe it can take it? I dunno...try scrolling up and down across the entire page repeatedly, very fast.[/citation]
I'm not on that machine right now, but on my AMD E350 netbook (also 4GB RAM@1066MHz, 320GB HD) with Windows 7 SP1 64-bit and Firefox 8, I tried scrolling up and down really fast and didn't notice anything. Actually, wait, there is a slight delay when I go by the top of the comments section. I have noticed that a little sometimes when scrolling through some webpages on here. But on my i5 machine, I don't notice it at all.
 
Didn't they come out with an extension on a RAM stick to a USB just last year....?? I don't remember the company that did it, but it's something like the same, just with a lot more options and a lot better.....
 
Status
Not open for further replies.

TRENDING THREADS

Latest posts

Moderators online

Share this page

COMPANY
RESOURCES
FOLLOW
Top Bottom