Does Cache Size Really Boost Performance?

[Wisecracker]

It looks like the article has a notable flaw...

* briefly puts on nerd cap *

The hypothesis being tested relates to cache ram size. The author did a good job making sure all 3 CPUs were clocked at the same speed. But it looks like the cache ram for each CPU was operating at 3 different speeds.

If I recall corectly, proper scientific method to test this requires that everything else be constant, and that the cache ram size be the only variable. While I don't exactly know how that could be achieved in this test, it would provide truly accurate results.

I have no doubt the differences in cache ram speed skewed the test results a bit. So the results may not be entirely accurate.

* removes nerd cap, stomps it flat, AND sets it on fire, AND extinguishes flames via 'natural' means *

What the crazy dawg in the funny geek hat said. Without accounting for the difference in L2 cache speed between the E2160, the E4400 and the X6800 any claims to cache size boosting performance is highly dubious.

 

[yomamafor1]

http://www.xbitlabs.com/articles/cpu/display/core2duo-e6420_3.html

 

[Slobogob]

@KyleSTL: lets try a little math...
For example, Quack IV:
174.6 / 167.7 = 1.0411 = 4.11% performance improvement between 2mb and 4mb

lets apply this to a game running at 35fps with a 2mb cache CPU, and we'll get 35 * 1.0411 = 36.44fps

Now, that's hardly an improvement that even deserves the time and attention I'm taking to write this reply. This won't realistically and noticeably improve choppiness.

In general i share your view on it, i have to point out that you are making an assumption. Applying your math would imply that the effect of L2 cache scales linear. To make such an assumption you would need more data.
In addition you can't directly compare the performance of Quake 4 with any other game.

That said, i still think you're right.

Still our point of view is based on subjective perception and speculation. What if the performance impact or L2 grows or declines stronger or weaker with clock speeds?

Another aspect, and the only point i'd like to criticize about the article, is the relation between L2 and system memory. Since the L2 is cache to save access time and reduce the amount of accesses it would be very interesting to see the effects of that.

If i, for example, compare a 4MB L2 processor using 800Mhz DDRII with a 1MB L2 processor using 800Mhz DDRII and set that data against a 4MB L2 processor using 533Mhz DDRII and a 1MB L2 processor using 533Mhz DDRII. The same set of tests could be done with DDR3 and DDR1 to see how the L2 effects the performance with faster or slower memory.
I would speculate that larger L2 cache somewhat can make up for slower memory - at least to a small degree. It is only speculation though.
In my eyes that is the only shortcoming of this otherwise pretty nice article.

 

[KyleSTL]

@eltoro: I see your point, but the difference between 1MB and 4MB is much greater (9.9% for Quake IV, 8.9% for Prey, and a 13.8% reduction in time for processing the WinRar bench). I realize the difference is not that great, but there is a difference, that's all I'm saying. We both have valid arguments. Let's leave it at that.

Also note that COD2 has no preference for a small or large cache. Future games may have similar performance, or maybe the games will have vast improvements due to the cache size, we don't know.

 

[SpinachEater]

The article is flawed and biased. I find it strange that the benchmarks that the author posted directly contradict his conclusion. One would think, that if the real world numbers were truly that much different, he would've posted those results, and not ones where it showed, in many cases, less that a 10% difference from top to bottom.

The benchmarks don't directly contradict the conclusion. The 1MB would have to outscore the 2 and 4 for that to be a direct contradiction. They just aren't as supportive as what they make them out to be, and so the biased conclusion comes in. When I saw the charts I was thinking...thats not a lot. 15 fps difference from 1MB to 4MB in Quake? Yes there was a boost but come on, that is splitting hairs. Keep in mind there is an error margin too because the tests are only so accurate. No one is going to go upgrade their CPU for 15 fps. Interesting study but the conclusion is unrealistic.

Oh yeah, quick calculation that they should have included / discussed -> Between the three games, the average FPS boost from 1MB to 4MB was only 9 FPS. That is, on average based on their study, 3 FPS / MB of cache increase. Sorry, I am only looking at FPS, I don't care about Divx that much

 

[spud]

@eltoro: I see your point, but the difference between 1MB and 4MB is much greater (9.9% for Quake IV, 8.9% for Prey, and a 13.8% reduction in time for processing the WinRar bench). I realize the difference is not that great, but there is a difference, that's all I'm saying. We both have valid arguments. Let's leave it at that.

Also note that COD2 has no preference for a small or large cache. Future games may have similar performance, or maybe the games will have vast improvements due to the cache size, we don't know.

What about UT3 and COD4?

Word, Playa.

 

[yomamafor1]

Don't know about COD4, but UT3 takes a significant advantage with larger L2 cache.

http://anandtech.com/video/showdoc.aspx?i=3127&p=2

 

[leo2kp]

Doesn't L2 run at core speed or FSB speed? :-/

 

[spud]

Doesn't L2 run at core speed or FSB speed? :-/

Core in this particular case later generations of CPU's it was 1/2 speed.

Word, Playa.

 

[KyleSTL]

@leo2kp & madfrog: I was pretty sure the L2 ran syncronously with the FSB or Bus speed or something, so if that's the case, then running all three processors in 1066FSB would make all cache speeds equal. Someone correct if I am wrong.

Edit, already answered, thanks, Playa.

 

[scryer_360]

Its well known that extra L2 equals better performance. Still, its also known that higher clock speeds equal better performance. You can have one or the other, but you cannot adjust L2, you can adjust clock speeds.

I'd rather get an E4400 and overclock it to 3.33 gigahertz than have a E6750 with 2 more megs of L2.

 

[russki]

all in all, I don't think anyone would argue that the relative effectiveness of cache is dependent on
1) architecture, e.g. effectiveness of branch predictions
2) its relative speed with respect to general purpose memory access speed.

The article did not address either of these, which I think prompted the bulk of the criticism here. Only a single architecture was examined, but even there the FSB speed was not fixed, as noted, whcih might influence the results.

Also, the code determines a lot of the effectiveness, too, which was not discussed in the article. In fact, that would be the explanation to the question why some applicatinos show more improvement than others.

By the way, to me, the largest area of concern is not that the authors did not test for all possible scenarios, but that they apparently have no understanding of the underlying concepts (maybe I'm wrong and they do, but they certainly don't explain it in the article)

 

[dakra]

Slobogob wrote "Since the L2 is cache to save access time and reduce the amount of accesses it would be very interesting to see the effects of that."

Yes, that is what cache is all about. Larger cache size mean fewer cache misses, until the memory working set fits in the cache. After that, an even larger cache doesn't help.

As the working set of Windows and the 50-100 ancillary processes running on your machine, besides your foreground application grows, you will feel the value of cache over raw processor speed.

According to my calculations:

For example, if a cache runs at processor speed, and a cache miss costs 50 processor waits, then going from a 99% hit rate to 100% will give a 49% gain in productive instruction cycles. A hit rate of 92% gives a 10% gain in productive cycles over a 91% hit rate.

If a cache miss costs only 25 processor waits, then a 100% hit rate gives 24% more productive cycles over a 99% hit rate. A hit rate of 92% gives an 8% gain in productive cycles over a 91% hit rate.

The benefits of large caches have been clearly observed when running J2EE web application server benchmarks.

Similarly, try your game benchmarks in the foreground while you are switched away from a large complex web page that constantly updates itself running java and javascript and reloads dozens of elements from the web (that it will find in browser cache on disk)

 

[SpinachEater]

I agree with you on that one Russki. Their conclusion was just a...Yep, it does...with a serious lack of relevant discussion. The magnitude of the performance boost wasn't even discussed...that is a huge no no. I envision interns writing that article for some reason...

 

[sirrobin4ever]

Once again, no AMD....

 

[drysocks]

Sheesh ...
Top three reasons you get what you get.
1. Money ( ie Money)
2. Money (ie Time)
3. Money (ie Material)

 

[SpinachEater]

AMD has one foot in the grave...let em die in peace.

 

[MrCommunistGen]

I don't know if this is valid or relevant, but I noticed when doing some video encoding on my rig and a friend's laptop C2D (1.8GHz/800MHz/2MB) that although his video encoding was a tad slower than mine, his system felt far more responsive. At the time I noticed it I proposed it was the extra cache. I'd be curious to see multitasking benchmarks.

-mcg

 

[russki]

the day hell freezes over will be the day mr schmid writes an article that peploe don't flame to hell.
...
seriously, though, in all my time visiting toms i have never seen one of his reviews get good support.

Stranger, you don't mean to imply there is some sort of a conspiracy at play here, do you? His articles are borderline passable, for the most part, at the top of the crop of the recent content here at Tom's. Cream of the crap, if you will.

You know, it used to be that Tom's was just as thorough if not more than Anand. Nowadays, this has been reversed in a big way.

I wasn't flaiming his article, really, but there is some content that is missing, you'd have to agree.

 

[kpo6969]

Once again, no AMD....

I second that.

 

[itotallybelieveyou]

Knew it! Cache never does anything!

 

[dragonsprayer]

nice article THG!
interesting the synthetic bench marks - show little difference
is this because synthetic tests are tuned to show results and are unreliable?
i often wonder about the synthetic tests - if they are really bench marks or just graphic shows?

o why no amd? amd is hurting bad enough at this point no reason to make them look even woarse? is that a flame?

 

[Evilonigiri]

nice article THG!
interesting the synthetic bench marks - show little difference
is this because synthetic tests are tuned to show results and are unreliable?
i often wonder about the synthetic tests - if they are really bench marks or just graphic shows?

o why no amd? amd is hurting bad enough at this point no reason to make them look even woarse? is that a flame?

Nah. It's just that you can't compare AMD's cache with Intel's, which you probably already know and he doesn't
Besides, AMD has fewer performance increases with larger L2 cache I believe.

 

[jimmysmitty]

Does anyone else here realize that comparing a Core 2 to a AMD K8 series would be just stupid? The K8 series is a roughly 5 year old architecture and is not compareable to the Conroe which is only a year old now. The Phenom would be compareable to Penryn/Conroe.

Besides Tom has the CPU charts where you can see the benchmarks and compare similar clocked CPUs with different cache sizes and so on.

I understand why you want to see AMDs results but this article was about seeing if cache makes a performance difference not whose CPU is better. I think it was interesting to see how real world test the cache did make a difference.

I can tell you cache size does make a difference. My old build had a P4 3.2GHz with 512k L2 cache(Northwood) and upgraded to a (Northwood again) P4EE 3.4GHz with 2MB L2 chace and my dual layer dvd ripping time from 30mins to 7mins. I haven't even tried my new system for dvd ripping yet although I am sure it will be uber fast compared to the old one.

Also if I remember correctly didn't Intel release a P4EE with a L3 cache as well? Was like 2-3 years ago but still.

 

[zenmaster]

AMD was not included because that would have been a different article.
This focused on the effect of L2 cache on Intel Processors.

It seems people flame just to flame or complain to just complain.
I have not seen any well thought out objection.

Note: The Cache Speeds are the same. (The closest thing to a valid argument, except it's just plain wrong.)

The following is from Intel's Website frm the description of one of the C2D Chips.


"L2 Cache Speed: The speed of the 2nd level cache. Since all current Intel® processors have internal L2 cache, the cache speed will be expressed in MHz/GHz or the speed in relationship to the processor core speed. For example, the Pentium® II processor and some early Pentium III processors had the L2 cache run at half the processor speed, while newer Pentium III processors and Pentium 4 processors have their cache run at the full speed of the processor. "