Is the ASUS crosshair V+580 giving biased results for the Bulldozer?

[mailpranshu]

Hi Guys,

I have stumbled upon something rather strange. Like any other builder I had been waiting for the FX with bated breath. It has been a disappointment and all sites across the net have thrashed it. Let me summarize some of the obvious pitfalls detailed on Toms, Anands, X bit etc.

1. Performs lower than the 2500k.
2. Power consumption is high.
3. Single core performance is dismal. Worse than even the 1100T in some cases!!!!
4. Gaming performance is downright bad.

I am from an engineering background(electronics) and have an additional finance background as well(MBA finance). I know how companies work and the pricing made no sense to me in any scenario. Let me summarize again.

1. Why Had AMD prices something so high when it knows it will not perform from day 1?
2. Whats the point of making a sure shot loss from day 1?
3. Why is the gaming performance so dismal? Wasn't ultra high resolution gaming its USP?
4. Where is the overclocking? 4.5ish GHz???? That's it???!!!!!!!!
5. Was there any testing done? Even a simple die shrink would have improved heating and efficiency.

So I wanted to have a further look. This was a mystery that needed looking into. I also noticed something else. All the testing on all the sites has been done on this kit. Link below.

http://morethanchessagame.forumotion.com/t1987-amd-fx-bulldozer-press-kit-pictured-stuffed-with-fx-8150-cpu-asus-crosshair-v-formula-and-scorpius-platform-badge

Now, all big sites have been given this kit including Toms. It makes sense because I am guessing Asus has an agreement with AMD to make its mother boards the "official" ones for testing. Also all these sites have used a 580 GTX quoting minimal graphics bottleneck. I agree its a sound rationale.

I confirmed at some other sites as well. The Asus + 580GTX has been used pretty much everywhere.

Its then that I went over to hardwareheaven and had a look. They have stayed away from the press kit and used an Asrock board. Here is the review link.

http://www.hardwareheaven.com/reviews/1285/pg1/amd-fx-8150-black-edition-8-core-processor-vs-core-i7-2600k-review-introduction.html

Now, Toms has come up with results exactly the same as all other sites that used the press kit. Lets have a look at some big changes on the benches.

Also, all sites except hardwareheaven has used a 580Gtx. Lets have a look at some gaming benches. That's what matters to us right?

3DMark 11

http://www.hardwareheaven.com/reviews/1285/pg9/amd-fx-8150-black-edition-8-core-processor-vs-core-i7-2600k-review-3dmark-11.html

Win

http://www.tomshardware.com/reviews/fx-8150-zambezi-bulldozer-990fx,3043-13.html

Massive loss!!!!!

F1 2011


http://www.hardwareheaven.com/reviews/1285/pg11/amd-fx-8150-black-edition-8-core-processor-vs-core-i7-2600k-review-f1-2011.html

Win

http://www.tomshardware.com/reviews/fx-8150-zambezi-bulldozer-990fx,3043-19.html

Massive loss!!!!!

Also when I go through the whole hardware heaven review it gives it a good rating placing it close to the 2600K and above the 2500K. Which in turn is in line with their strategy. It does lose in some benches but is still very very close to the 2500K.

This is extremely strange. Please go through the site yourself.

So to conclude:
1. Can Toms and others please do a test with other 990 boards?
2. Can we please have a comparison with more AMD and Nvidia cards?
3. Who do we trust? Isn't a biased review completely anti against AMD from day 1?
4. Can some one else independently run benches with the 2500K, 2600K and 8150 using different cards and mother boards?
5. Is there a Nvidia Intel nexus to make sure 8150 looks bad? Was the BFBC3 patch doctored for more than just RAGE???( remember Hawx 2 tessellation)

Please post links here. And don't post links for other sites as there is a strong anti AMD bias/ vibe. I think we users need to take things in our own hands and check this out. Can some one use their own personal hardware and run benches?

I am not an AMD fan or a conspiracy nut. I use both builds and use Nvidia as well. So lets keep this discussion open and friendly. No trolling please.

And can the seniors moderators and testers at Toms please step in and explain? Can a new test be done?

 

[Cazalan]

Looks like a major difference isn't neessarily the motherboard, but this

2x 4GB Corsair Vengeance DDR3-1866

Yes I pointed that out in another thread. WHY did these sites use DDR3-1600 when FX officially supports 1866. I think they wanted to keep system specs as close as possible to Intel but that's not right.

As seen with Llano that 1600 to 1866 jump is significant.

 

[3Ball]

Hard OCP used an MSI Board and a 6970.

http://www.hardocp.com/article/2011/10/11/amd_bulldozer_fx8150_gameplay_performance_review

http://www.hardocp.com/article/2011/10/11/amd_bulldozer_fx8150_desktop_performance_review

Best,

3Ball

 

[AdrianPerry]

Hard OCP used an MSI Board and a 6970.

http://www.hardocp.com/article/2011/10/11/amd_bulldozer_fx8150_gameplay_performance_review

http://www.hardocp.com/article/2011/10/11/amd_bulldozer_fx8150_desktop_performance_review

Best,

3Ball

Good results there from BullDozer - closed to what I expected it would be, but once again it shows another hardware config is displaying totally different results to others.

 

[Rizlla]

This is really interesting. I am keeping tuned.

 

[szymek]

Well, it's not gtx580 that's making it, thats for sure.
http://pclab.pl/art47038-4.html
http://pclab.pl/art47038-5.html
http://pclab.pl/art47038-10.html
Here are tests done with 6970. Must be the damn Asus board. Even though I own i7 2600k (switched from pII 1090t only because I got intels rig for free) I really would like seeing AMD back in game and kicking Intel's ass.

 

[obsidian86]

i think there might be another explanation

http://www.xtremesystems.org/forums/showthread.php?275786-AMD-FX-8150-Bulldozer-finally-tested&p=4969164&viewfull=1#post4969164

there seems to be an l1 cache bug

 

[cangelini]

Hi Guys,

I have stumbled upon something rather strange. Like any other builder I had been waiting for the FX with bated breath. It has been a disappointment and all sites across the net have thrashed it. Let me summarize some of the obvious pitfalls detailed on Toms, Anands, X bit etc.

1. Performs lower than the 2500k.
2. Power consumption is high.
3. Single core performance is dismal. Worse than even the 1100T in some cases!!!!
4. Gaming performance is downright bad.

I am from an engineering background(electronics) and have an additional finance background as well(MBA finance). I know how companies work and the pricing made no sense to me in any scenario. Let me summarize again.

1. Why Had AMD prices something so high when it knows it will not perform from day 1?
2. Whats the point of making a sure shot loss from day 1?
3. Why is the gaming performance so dismal? Wasn't ultra high resolution gaming its USP?
4. Where is the overclocking? 4.5ish GHz???? That's it???!!!!!!!!
5. Was there any testing done? Even a simple die shrink would have improved heating and efficiency.

So I wanted to have a further look. This was a mystery that needed looking into. I also noticed something else. All the testing on all the sites has been done on this kit. Link below.

http://morethanchessagame.forumotion.com/t1987-amd-fx-bulldozer-press-kit-pictured-stuffed-with-fx-8150-cpu-asus-crosshair-v-formula-and-scorpius-platform-badge

Now, all big sites have been given this kit including Toms. It makes sense because I am guessing Asus has an agreement with AMD to make its mother boards the "official" ones for testing. Also all these sites have used a 580 GTX quoting minimal graphics bottleneck. I agree its a sound rationale.

I confirmed at some other sites as well. The Asus + 580GTX has been used pretty much everywhere.

Its then that I went over to hardwareheaven and had a look. They have stayed away from the press kit and used an Asrock board. Here is the review link.

http://www.hardwareheaven.com/reviews/1285/pg1/amd-fx-8150-black-edition-8-core-processor-vs-core-i7-2600k-review-introduction.html

Now, Toms has come up with results exactly the same as all other sites that used the press kit. Lets have a look at some big changes on the benches.

Also, all sites except hardwareheaven has used a 580Gtx. Lets have a look at some gaming benches. That's what matters to us right?

3DMark 11

http://www.hardwareheaven.com/reviews/1285/pg9/amd-fx-8150-black-edition-8-core-processor-vs-core-i7-2600k-review-3dmark-11.html

Win

http://www.tomshardware.com/reviews/fx-8150-zambezi-bulldozer-990fx,3043-13.html

Massive loss!!!!!

F1 2011


http://www.hardwareheaven.com/reviews/1285/pg11/amd-fx-8150-black-edition-8-core-processor-vs-core-i7-2600k-review-f1-2011.html

Win

http://www.tomshardware.com/reviews/fx-8150-zambezi-bulldozer-990fx,3043-19.html

Massive loss!!!!!

Also when I go through the whole hardware heaven review it gives it a good rating placing it close to the 2600K and above the 2500K. Which in turn is in line with their strategy. It does lose in some benches but is still very very close to the 2500K.

This is extremely strange. Please go through the site yourself.

So to conclude:
1. Can Toms and others please do a test with other 990 boards?
2. Can we please have a comparison with more AMD and Nvidia cards?
3. Who do we trust? Isn't a biased review completely anti against AMD from day 1?
4. Can some one else independently run benches with the 2500K, 2600K and 8150 using different cards and mother boards?
5. Is there a Nvidia Intel nexus to make sure 8150 looks bad? Was the BFBC3 patch doctored for more than just RAGE???( remember Hawx 2 tessellation)

Please post links here. And don't post links for other sites as there is a strong anti AMD bias/ vibe. I think we users need to take things in our own hands and check this out. Can some one use their own personal hardware and run benches?

I am not an AMD fan or a conspiracy nut. I use both builds and use Nvidia as well. So lets keep this discussion open and friendly. No trolling please.

And can the seniors moderators and testers at Toms please step in and explain? Can a new test be done?

Solid analysis there, mailpranshu.

For background, the Asus board was used because that was the platform validated by AMD and shipped out with its press kits. It's basically the board where they say, "We've tested this in the lab, we know it works as-expected; we can't say the same about any other board right now." From that perspective, choosing another platform instead for the launch could have potentially been problematic, and at that point, there's no fall-back. When you stick with what's validated, you have AMD's assurance and lab time behind it.

Now, I can assure you that, if there's something going on with the motherboard, we'll be getting to the bottom of it soon, as part of our plan for launch was to also have a high-end 990FX board round-up ready to go. Thomas is working on that, and should a performance difference between boards emerge, it'll be clear right off the bat, I imagine. He'll have ASRock, Asus, Gigabyte, and MSI all represented.

As far as comparing graphics cards, I can actually handle that one in the next day or so (drop in a 6970, run some of the game tests, let you know what happens).

For what it's worth, I took a leap of faith and sent all of my results/charts to AMD nearly a week before the review went live, just for a reality check. AMD's only feedback was that I needed to use newer applications to demonstrate the performance of FX. In my story, I'm using the latest versions of every app I could test with (aside from SolidWorks, which I believe is up to 2012 now). AMD didn't flag any of the results as unexpectedly low, nor did it raise an eyebrow about the GTX 580 and suggest one of its cards would perform better. Thus, I'm extremely confident in the accuracy of my benchmark results insofar as they correlate to what AMD was expecting to see.

Until I have more to report,
Chris

 

[torque79]

Excellent reply mr chief editor. I'm REALLY glad to see tom's is on top of this confirming their results. I'm inclined to agree that your results are going to be shown as typical, but I would be thrilled to be proven wrong. After seeing these benches I was so angry after waiting so long for bulldozer I already ordered a i5 2500k kit and gave up on AMD this time. I am totally an AMD fanboy but after such huge delays, if it's just a release day fiasco due to bios flaws or a L1 cache bug, that's going to REALLY frustrate me. Probably not enough to ship back my intel and mobo, unless the turn around is REALLY impressive.

 

[se7envii]

Awesome thanks for the response Chris. I'll be looking forward to those articles. I see potential in these FX chips but I'm still probably going to go with the 2600K unless I see evidence that these FX chips can unleash their full potential in more everyday workloads.

 

[cangelini]

Just ran the test that always demonstrates the most disturbing result: F1

1680x1050, Ultra Quality, no AA: 58.44 FPS Avg (Radeon HD 6970)
1680x1050, Ultra Quality, no AA: 61.5 FPS Avg (GeForce GTX 580)

Will try to do some more in a bit!

 

[-Fran-]

Uhm... Mr. Chris... You sent the results to AMD a few weeks ago, but the 0813 BIOS revision from Asus has been avaliable from 2011/10/07.

Should I assume you guys had the BIOS way before the public release? Maybe a beta version of it?

Cheers!

 

[szymek]

Mobos with updated bios usually are shipped before being given to public. I got my mobo with a bios that was released to the public a month later.

 

[-Fran-]

But the Crosshair V has been shipping since a long long time now. Mine came with the 04xx revision. Now it has the 07xx since 0813 is for FX.

Cheers!

 

[cangelini]

Uhm... Mr. Chris... You sent the results to AMD a few weeks ago, but the 0813 BIOS revision from Asus has been avaliable from 2011/10/07.

Should I assume you guys had the BIOS way before the public release? Maybe a beta version of it?

Cheers!

Not a few weeks ago, *last* week I had a final version of the BIOS installed, from Asus, prior to testing.

 

[-Fran-]

Oh, nice. Thanks for the answer.

Keep up the good work.

Cheers!

 

[cangelini]

Very welcome Yuka, thanks for reading! =)
Chris

 

[jdw_swb]

Here is another review using the Asrock Fatal1ty 990FX board.

I'll post some of the benchmarks. Looks like using the ASROCK board still shows bad results for Bulldozer.

 

[jonnyrb]

http://blogs.amd.com/play/2011/10/13/our-take-on-amd-fx/

Playing the Latest Games

A perfect example is Battlefield 3. Take a look at how our test of AMD FX CPU compared to the Core i7 2600K and AMD Phenom™ II X6 1100T processors at full settings:

Map Resolution ----------------------AMD FX-8150, Sandy Bridge i7 2600k, AMD Phenom™ II X6 1100T
MP_011 1650x1080x32 max settings 39.3 37.5 36.3
MP_011 1920x1200x32 max settings 33.2 31.8 30.6
MP_011 2560x1600x32 max settings 21.4 20.4 19.9
Benchmarking done with a single AMD Radeon™ HD 6970 graphics card

 

[cuecuemore]

Did anyone notice the Sandra performance was way worse on Hardware Heaven than any other site? I don't own any of the games they tested, but it looks like they're all multi-threaded, perhaps optimized for more than 4?

 

[torque79]

Don't those results you posted show the 8150 beating or matching the i72600k at 1920x1200 resolution half the time? The losses are by a wider margin, but I would argue that's likely due to either/both of:
- game is optimized for the other gpu/cpu manufacturer
- game is not well programmed for multi-threaded use (at least past 2-4 threads)

I think it may actually be largely software being poorly designed for multithreading that's preventing us seeing the 8150's real potential. Unfortunately we DO want to play with games currently on the market, but this leads me to believe that in a year or so the 8150 will be on par with the i72600k. Unfortunately, by then ivy bridge will be out, but then we'll have the ongoing back and forth between ivy/piledriver and those that come next.

 

[Jprobes]

Don't those results you posted show the 8150 beating or matching the i72600k at 1920x1200 resolution half the time? The losses are by a wider margin, but I would argue that's likely due to either/both of:
- game is optimized for the other gpu/cpu manufacturer
- game is not well programmed for multi-threaded use (at least past 2-4 threads)

I think it may actually be largely software being poorly designed for multithreading that's preventing us seeing the 8150's real potential. Unfortunately we DO want to play with games currently on the market, but this leads me to believe that in a year or so the 8150 will be on par with the i72600k. Unfortunately, by then ivy bridge will be out, but then we'll have the ongoing back and forth between ivy/piledriver and those that come next.

^^ I wholeheartedly agree.

If in a year a lot of the new games that come out, BF3, Skyrim, ME3 and whatnot are optimized for multithreading giving credence to their forward thinking approach to the FX design, the PD benefit even more from game design moving in this direction.

I believe that there are a few factors that play into the launch benchmarks beyond just stamping dud on it while standing on a pedestal screaming as much.

the FX is a beast when it comes to multithreaded app's that cannot be denied. I ordered 3 for workstations (Seismic Engineering\CDocs Creation).

People are so quick to cast judgements without taking the time themselves, or letting time pass to get an accurate assessment of where this new tech fits into today's environment. I believe it is ahead of its time and it will flourish once the software industry moves further down the field towards what it was designed for.

On a side note, I do not understand the obsession with benching games at such low resolution to prove a point about which processor is better then the other.

That is like basing a purchase of a vehicle on how fast it goes in 1st gear, or comparing the gas mileage of cars by driving them at 20mph until they run out of gas to see which one is better.

I would like to see realworld benchmarks in games while somewhat taxing the system. I know that when i play games I have a lot of other things running in the background like Spybot, MWB, Steam, Chrome/Firefox with several tabs open on a second monitor, Skype and IM, Bluebeam/adobe, Outlook & Excel, Flash plugins (I tend to keep ESPN open following games on gametracker).

A benchmark with those things is far more beneficial to my decision making then lowering the resolution to 800x600 to see which one has the biggest wang after a cold shower.

We live in a fickle world where we crave instant gratification. When that moment comes and we are not gratified to the extent that we deemed sufficient we lash out and become bitter in an attempt to self gratify through cognitive dissonance.

 

[Onus]

This is a very interesting thread...will be following it. It does seem a little like there's some grasping at straws going on, but some of those straws may very well be real. A L1 cache bug could be huge.
Something to remember though, I'm not sure running a whole bunch of single-threaded apps is much like running a single multi-threaded app; even if the number of threads ends up similar, but I would like to be corrected about that if I'm wrong.

 

[unity100]

that explains a lot of things.

sadly, i bought one of the gigabyte 990fx boards, which seem to flop with bulldozer.

since asrock and msi boards performed better to the point of making bulldozer come out ahead with the 990 chipset, i assume that the issue is a bios issue. i hope gigabyte and asus fix their bioses asap.

 

[warmon6]

This is a very interesting thread...will be following it. It does seem a little like there's some grasping at straws going on, but some of those straws may very well be real. A L1 cache bug could be huge.
Something to remember though, I'm not sure running a whole bunch of single-threaded apps is much like running a single multi-threaded app; even if the number of threads ends up similar, but I would like to be corrected about that if I'm wrong.

agree. [:grahamlv:3]

 

[dontqqnub]

Still waiting for response from other tomshardware mods and chris!