BAPCo Bites Back at AMD's Departure

[gto127]

I agree with AMD'S decision to leave. Since the benchmark only comes out about every 5 years it should include software that will test the most updated scenarios such as GPU processing which is where processing is headed.

 

[justjc]

woops forgot Office that is tested as 2010, where the new Office 11 also supports the use of the GPU to lighten the workload for the CPU.

If nothing else all these examples show how out of date a benchmark not giving the GPU any value is today and I almost don\t dare to think how many of the programs on the list at http://www.bapco.com/products/sysmark2012/applications.php will also have been switched with GPU accelerated versions in the year this bnchmark is supposed to represent by its name.

A clear fail, most likely due to BAPCo still favoring to Intel who would loose greatly if GPU friendly languages like OpenCL entered the picture. .

 

[justjc]

[citation][nom]justjc[/nom]woops forgot Office that is tested as 2010, where the new Office 11 also supports the use of the GPU to lighten the workload for the CPU. If nothing else all these examples show how out of date a benchmark not giving the GPU any value is today and I almost don\t dare to think how many of the programs on the list at http://www.bapco.com/products/sysm [...] ations.php will also have been switched with GPU accelerated versions in the year this bnchmark is supposed to represent by its name.A clear fail, most likely due to BAPCo still favoring to Intel who would loose greatly if GPU friendly languages like OpenCL entered the picture. .[/citation]

Somehow my previous message never posted...
I only pointed out that in the most used parts of the everyday experience, surfing the net, the software chosen somehow always was the last version before the software got GPU acceleration.

They choose IE8, where IE9 uses the GPU
and Firefox 3.6.8, where 3.7 was renamed Firefox 4 and use the GPU
finally they decided to go with Adobe Flash 10.1, where all later versions have GPU acceleration.

 

[fir_ser]

If AMD SYSmark is unfair for AMD’s processors then I agree with its decision in leaving the consortium .

 

[Guest]

@otacon72

Given the right conditions a car could appear to be faster then an airplane (accelerating over 100m), must make you feel proud to believe your car is faster then an airplane.....

ps i think you missed the article about cracking an encrypted file with GPU assistance, junk indeed

 

[guyjones]

I comment as an AMD user -- the company would do well to worry less about benchmark standards that may or may not reflect well on its products and more about getting said products to market in a timely fashion. The company is already one product cycle behind Intel's Sandy Bridge offerings, and the technology gap only seems to be getting wider. You can't compete in an industry if consumers don't even have a choice of whether to buy your products. It would be one thing if AMD had some substantial mobile presence to offset Intel's desktop dominance, but it doesn't.

AMD is skating on very thin ice right now. It has very little margin for error on the path which it is treading.

 

[BSMonitor]

Read the initial comments. The statement by BAPCo clearly says they supported 100% of the initiatives AMD put forth. And AMD voted in favor of 80% of the components of BAPCo. Yet AMD donkey's respond by saying BAPCo is out of line?!??

LMFAO at AMD Fanboys.

Plain and simple. K10 cores get dominated clock for clock by Conroe/Penryn processors. Nehalem and Sandy Bridge, well its really a joke comparing them to K10 or K10.5 or any K core around.

AMD clearly doesn't like how weakly their new CPUs perform and wanted benchmarks that Sandy Bridge dominates thrown out. Sorry AMD, throwing more crappy cores at a single or dual-threaded App doesn't cut it. Intel's turbo on dual and quad core setups make AMD 6-8 core chips look like jokes.

 

[Guest]

well at 700.00+ per copy who cares any way?

 

[verbalizer]

otacon72 is just that, a 'con'..

 

[Guest]

"BAPCo is a non-profit consortium made up of many of the leaders in the high tech field, including Dell, Hewlett-Packard, Hitachi, Intel, Lenovo, Microsoft, Samsung, Seagate, Sony, Toshiba and ARCintuition," the company said in an emailed letter. "

Intel....says it all. It may not be fair software.

 

[ajcroteau]

Explains why Intel is sooo much more expensive than AMD. Can you say... conspiracy...?

 

[cookoy]

in the end, we will see different benchmark results, and those with biased results will be exposed and discredited. who will guard the guards? who will benchmark the benchmarks?

 

[silverblue]

[citation][nom]BSMonitor[/nom]LMFAO at AMD Fanboys. Plain and simple. K10 cores get dominated clock for clock by Conroe/Penryn processors.[/citation]
This again? I've already disproved this - in most situations, aside of anything specifically optimised for Intel architectures, there's very little, if any difference, and in some cases K10.5 is faster. Go and find me a Core 2 Quad that beats the best Stars CPU across the board and you may indeed have a point, but until then, I'm not going to take you seriously.

 

[silverblue]

I'll add a little more weight to my reasoning so as to avoid being labelled as an AMD fanboi, which no doubt will happen.

The Penryn architecture benefits from extra L2 cache. Compare, if you will, the 9400 (2.66GHz, 6MB L2) to the 9450 (2.66GHz, 12MB L2 cache). Same core, same FSB, just more cache.

http://www.anandtech.com/bench/Product/76?vs=51

In any test that works well with lots of cache (and with L2, it's much faster than going to L3), the results are up to 19% faster in places, but on average, the gains are not substantial. Note how much the SYSMark 2007 - Overall result is skewed by the Productivity score (the very same 19% I mentioned earlier). Intel CPUs utilise an inclusive cache system, and as Core 2 Quads are two Duos bolted together, this would in essence point to 6MB that neither package has to share with the other. AMD CPUs employ an exclusive architecture which allows them to use less cache, however there's a sizeable penalty incurred in looking for data. Throwing more L3 cache at the Phenom II was a big help but it's far slower than having a large area of L2. Notice that Llano uses 1MB of L2 per core and enjoys an IPC advantage over previous K10.5 designs under most workloads even without having any L3 cache at all. I'm sure L3 helps in some circumstances, though.

So, let's change the comparison...

http://www.anandtech.com/bench/Product/81?vs=76

For obvious reasons, let's ignore SYSMark.

CS4 - traditionally strong on Intel CPUs. 18% faster.
DivX - Intel 7% quicker
x264 - 1st Pass - AMD 4% quicker; 2nd pass - Intel 8% quicker (AMD never do well here)
WME - no difference
3dsmax r9 - too many to calculate but more wins for Intel than for AMD, not a large difference overall
Cinebench R10 - tiny win for AMD, could be within the realms of natural variability between tests
POV-Ray 3.7 Beta 23 - small win for Intel
PAR2 - AMD 5% quicker
Blender 2.48a - Intel 23% quicker, big win
Monte Carlo - Intel 42% quicker, traditionally strong test for them/weak for AMD
Sony Vegas Pro 8 - AMD 3% quicker
Sorensen Squeeze Pro 5 - Intel 8% quicker
WinRAR 3.8 Compression - AMD 16% quicker
Games average - Intel 5% quicker (Far Cry 2 - an amazing 27% quicker for Intel, but the others show a negligible difference (small AMD win in Fallout 3))

The 920 has a 133MHz advantage plus the "benefit" of L3 cache, however there's little evidence that really helps here for most tests. More L2 cache than 512KB per core may have been a lot more useful.

You'll notice that Intel doesn't sweep the board. Yes, Core 2 is faster, per clock, in most scenarios, but not noticably, and certainly not dominated as you claim. Far Cry 2 was developed by Ubisoft in conjunction with Intel. AMD seems to be very good in WinRAR and relatively decent overall at video compression, but falls down with Excel 2007, Blender and CS4 which also appear to be optimised very well for Intel architectures or Stars is just plain bad at them.

In the end, there's very little difference between Penryn and K10.5 at similar clock speeds for the majority of tasks (albeit with a limited testing suite to go off), and if a processor does something rather well, it makes more sense to buy that one. If you want a more rounded CPU, the Core 2 Quad would make more sense in this circumstance, but far too often you'll find that they're priced far too much, definitely so for the 12MB L2 cache models.

Time for an unfair comparison:

http://www.anandtech.com/bench/Product/146?vs=48

A $1,000 (at least) QX9770 being beaten in most tasks by a 1090T, but like I said, it's unfair as the AMD part has 6 cores and Turbo Core. IPC aside, in this case it'd be madness to own any Core 2 product for productivity, but the Core 2 Quad is a more consistent performer across the board.

Neither you nor I truly know how Bulldozer is going to perform yet.

 

[sdaccount]

Here is an interesting article:

http://www.brightsideofnews.com/news/2011/6/24/amd-insiders-speak-out-bapco-exit-is-an-excuse-for-poor-bulldozer-performance.aspx

Some quotes:

"When I read Nigel's blog and saw the press release from BAPCo it made me sick because our CMO talks about transparency and honesty and it's all smoke and mirrors. At the end of the day, we actively had internal teams and external organizations hired to promote/discredit SYSmark. Not because it was inaccurate, but because it is accurate. Back in the original Athlon 64 and Opteron days, when we were winning in SYSmark we were heavily promoting it in the public sector, who in turn used it as a benchmark on which they based many of their purchases on. It was us who actually got BAPCo and SYSmark inside several government tenders to win orders measured in tens of thousands of systems. SYSmark was used to show how our K8 processors were beating Intel's NetBurst."

"The minute we started losing performance leadership, which was Woodcrest/Conroe/Merom timeframe was when we started to put together active groups that would discredit SYSmark."

"A shift occurred and all engineers were left was not making a discussion that would go along the lines of "ok, how do we make our architecture better?", "how do we work to regain performance leadership?". According to one of our highly positioned sources, the culture switched to "how do we discredit benchmarks and skew the numbers?" "

""Because Bulldozer performance is sub-par. Because after such a long wait Bulldozer will not be a world beater. They knew that for another generation, we would come up way short on benchmarks like SYSmark. Removing ourselves from BapCo was the best way to make the benchmark look like it's no longer valid in the hope that prospective or existing customers stop using it.""

""Bulldozer is going to disappoint people because we did not get the resources to build a great CPU, and it's not that we needed billions of dollars to make it a leader. We needed investment in people, tools and technology.""

"Our sources went on to say that the launch of Llano clearly shows what is the current and future strategy - downplay CPU performance every chance you get. Everything has to revolve around the GPU."

 

[silverblue]

There's something important about the article you've mentioned. It specifically attacks the Bulldozer "core". The thing is, we already knew that a single Bulldozer core will not significantly outperform a Llano core. Through the Anandtech preview article, we were told that AMD had reduced the number of ALUs from 3 in Stars to 2 per Bulldozer core to reduce core size but as there have been improvements made to the design, this wasn't an issue, and Bulldozer should beat Stars for single-threaded performance. Bulldozer packs two cores into a Bulldozer module. For each Orochi/Zambezi die, we're talking four modules/eight cores in comparison to a Llano CPU which has a maximum of four cores, and whether Bulldozer performs badly per core or not is going to be largely irrelevant - Bulldozer will still easily outperform Llano in a good deal of situations, integer ops being the most obvious example. The source of that article's information isn't actually telling us anything we don't already know. Something else we already know is that for AVX functions, Bulldozer will trail SB because SB has more resources for the task, but as AVX functions aren't being adopted too quickly, and because Bulldozer can still do them, it's not as if AMD are releasing a CPU without proper support for today's instruction sets, one of the biggest complaints I had for Stars.

I find it amusing that AMD "started to spend resources on how they could skew benchmarks in their favor" but the article started off by saying "As you can read here, the source alleges that AMD first had internal teams and organizations in order to promote SYSmark, and then turned those teams and organizations to discredit the benchmark when it didn’t work in their favor.". How can they have done both? If they had opted to skew benches in their favour, surely we would've seen that to start with? If AMD devoted resources to discrediting BAPco, why have we not heard of this before? AMD were outperforming P4 because they were outperforming P4, not because they cheated in a single benchmark suite.

I think it's very likely that Bulldozer's per-core performance will be disappointing and will definitely show up in SYSMark 2011. Yes, this might be why they left. I'm going to argue that Bulldozer is designed for multithreading first and foremost, hence the aggressive implementation of CMT. Until there's actual figures out there, we're not going to know. It's likely that such a CPU is not suited terribly well to gaming as it has limited FPU resources, but they're still going to exceed Stars and, as AMD has said time and time again, Bulldozer is meant to be paired with a discrete card if you're going to do anything that requires FP calculations, so is this deficiency actually as large an issue as we're being lead to believe? Have AMD got it wrong? We don't know at all and won't for another two or three months. Bulldozer is looking to be more of a server product than something you would use at home, nevertheless it'll still perform much better than Stars.

"When asked about core performance, surprising information was that a Bulldozer core versus the existing cores in Llano will result in minimal improvements overall. Our sources went on to say that the launch of Llano clearly shows what is the current and future strategy - downplay CPU performance every chance you get. Everything has to revolve around the GPU."
If a system has a GPU, that GPU should be tested. If it's not being factored in (or, at least, not sufficiently), is that a fair reflection of system performance? Like it or not, GPU performance does actually matter, and whether AMD is right or wrong for wanting this taken into account is irrelevant. Bulldozer's pairing with a discrete GPU may bring its score up significantly as it's supposed to be paired with one and AMD are likely to be very upset that the obvious shift towards a focus on graphics performance may be going unnoticed. AMD aren't the only people trying to bring CPUs and GPUs closer together, which means that SYSMark should take this into account.

The fact is, AMD don't need to downplay CPU performance. In most cases, it's perfectly adequate. Until recently, you didn't need more than a dual core for gaming. Turbo modes are an extra help to ensure that even a modest CPU can give you the performance you need pretty much all the time and especially in situations where software cannot take proper advantage of extra cores. Both Intel and AMD know this. Tech site after tech site are subscribing to the theory of "good enough performance" for non-enterprise CPUs, so it's not as if AMD are forcing people to change their thinking.

Bulldozer's delay might simply be down to getting a fast enough chip out to make up for its deficiencies in lightly-threaded apps as reported with the B0 and B1 steppings. We already know it's been designed for high clock speeds. Theo Valich cannot suddenly believe that Bulldozer is a poor chip as he himself said he expects it to be a number-crunching monster. If AMD have said Interlagos is 50% faster than Magny Cours at integer tasks (8m/16c/16t vs. 12c/t) then it's clear that isn't not going to perform like a dog under that usage scenario unless it's just a figure plucked out of the air. The main issue about all of this is whether AMD is lying; people would say that lack of actual benchmark results is evidence enough of their guilt. We don't know.

All this is speculation; Bulldozer's actual performance, who this insider is and his role at AMD, and whether Nigel Dessau was lying about AMD's withdrawal from BAPco. One thing to note is that NVIDIA's departure cannot be a mere coincidence.