ATi cheated in 3DMark2001SE too(figures)

[GeneticWeapon]

Alexy, maker of Rivatuner, uncovered Nvidia cheating in 3DMark2001SE(as some of you know) by designing his Rivatuner program to stop shader detection in apps such as 3DMark. He claimed to be testing his program on an R300 class board next, so I patiantly waited to see what he would find. Unfortunately, he found this....

Unwinder:
Catalyst 3.4

3DMark Score 11676
Game 1 - Car Chase - Low Detail 167.6 fps
Game 1 - Car Chase - High Detail 61.3 fps
Game 2 - Dragothic - Low Detail 206.8 fps
Game 2 - Dragothic - High Detail 119.7 fps
Game 3 - Lobby - Low Detail 157.2 fps
Game 3 - Lobby - High Detail 71.1 fps
Game 4 - <b>Nature 66.0 fps</b>
Fill Rate (Single-Texturing) 961.3 MTexels/s
Fill Rate (Multi-Texturing) 2160.8 MTexels/s
High Polygon Count (1 Light) 54.8 MTriangles/s
High Polygon Count (8 Lights) 13.6 MTriangles/s
Environment Bump Mapping 129.5 fps
DOT3 Bump Mapping 128.0 fps
Vertex Shader 149.6 fps
Pixel Shader 199.6 fps
Advanced Pixel Shader 149.4 fps
Point Sprites 27.0 MSprites/s

Ñatalyst 3.4 + ATIAntiDetector

3DMark Score 11291
Game 1 - Car Chase - Low Detail 165.4 fps
Game 1 - Car Chase - High Detail 63.2 fps
Game 2 - Dragothic - Low Detail 207.5 fps
Game 2 - Dragothic - High Detail 119.3 fps
Game 3 - Lobby - Low Detail 155.5 fps
Game 3 - Lobby - High Detail 70.8 fps
Game 4 - <b>Nature 47.1 fps</b>
Fill Rate (Single-Texturing) 963.4 MTexels/s
Fill Rate (Multi-Texturing) 2154.2 MTexels/s
High Polygon Count (1 Light) 54.0 MTriangles/s
High Polygon Count (8 Lights) 13.7 MTriangles/s
Environment Bump Mapping 129.2 fps
DOT3 Bump Mapping 128.0 fps
Vertex Shader 149.1 fps
Pixel Shader 199.4 fps
Advanced Pixel Shader 149.6 fps
Point Sprites 27.0 MSprites/s

AT least 3 pixel shaders are detected (1@1.1 and 2@2.0), texture detections (thats how GT4 in 3dmark2001 is detected) ....
Full texture patterns& shader code are in the driver, but although easy to see what they do, its very hard to make FULL anti-detect patch.....
.... what ATi make with these 2 p.sh. 2.0 is hardly "shifting" ......

You need to look at the scores of Game Test 4 'Nature' in both tests. 3DMark2001SE is a test of both DX8, and DX8.1 functionality. GT4 is asking for DX8 pixelshader 1.0, and ATi's Catalyst drivers are detecting it and delivering DX8.1 ps1.4 for R200 based cards, and DX9 ps2.0 for R300 based cards. They are supplying a faster driver to do the job, yes,....but they are using a method of application detection stored in their drivers to do the swap. Regardless, 3DMark is calling for pixelshader 1.0, not 1.4, or 2.0. I would probably get by with calling it an optimization if they werent using a detection method to achieve it. But they are.
Although the score in the Nature test did not jump as high as Nvidia's did, I'm pretty bummed now, knowing that the benchmarks I have been relying on the LAST TWO YEARS were skewed by BOTH companies.

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>120% Myself!</font color=red>

 

[rubikian]

Better use the rest of the game benchmark then...

 

[rubikian]

Do they detect the rest of the game benchmark as well?

 

[GeneticWeapon]

Only the Nature test has been violated....

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>120% Myself!</font color=red>

 

[Willamette_sucks]

r300 based board eh? 9700 Pro?
What were the other specs?

cpu at 200*12 and 9800 pro at 420/360 I ran the nature test, and it NEVER dropped below 100fps (what a happy sight to see), and the avg was probly 120-130 or so.

Neways, yeah, that sux that ATI was cheating too.
With Rage 3d tweak, there is an option to FORCE a pixel shader version (assuming the card supports it). Im not sure if it works, but im definately gonna try it out.
Put it on Force 1.0 and run the nature test.
Ill let ya know what if anything happens.

"Every Day is the Right Day." -Pink Floyd

 

[GeneticWeapon]

I dont know if that would work Will, you would be 'forcing' ps 1.0 on all the 3DMark tests, it probably wont run the benchmark. The advanced pixelshader test is going to call for PS 1.4 from your card, and if PS 1.0 is given instead, it might crash or something.

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>120% Myself!</font color=red>

 

[Willamette_sucks]

I ONLY ran the Nature benchmark when I did this.

Neways, I tried it, but I dont think the tweak had any effect because I got the same score on each run.

I think the tweak is only for increasing the ps version the card uses (such as ps2 vs ps1.4 when available) if for some reason the driver defaults are messed up and set to 1.4 or something.

Too bad, I was hoping to at least try it out.

Theres also a tweak to force the Vertex shader version.

vertex shaders are also used in the nature benchmark, do you think that would have an effect? (not the tweak, as it may not work, but if you could actually lower the vertex shader version)

But if he didnt detect a vertex shader enhancement then I guess not.

"Every Day is the Right Day." -Pink Floyd

 

[GeneticWeapon]

Well, there are only 2 different vertex shaders out there, and neither one would provide a boost in score if they were swapped with one another...dunno.

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>120% Myself!</font color=red>

 

[phial]

im NOT trying to be an ATI fan

but, if the hardware has the capability to do something exactly the same with less work (ps1.4 vs ps1 or 1.1) then why not ? it doesnt lower the IQ, doesnt do any tricks that wouldnt be applicable to games right? this to me is totally different than what Nvidia did. its an optimization (which arent bad things!) not a cheat or workaround

i think it was incredibly smart for ATI to make the 8500 ps1.4 hardware. because of this it is still able to compete with the TIsereies and the low end GF5200. even the 9000pro renders pixel shaders faster than the GF5200 ffs



-------

<A HREF="http://www.quake3world.com/ubb/Forum1/HTML/001355.html" target="_new">*I hate thug gangstas*</A>

 

[Willamette_sucks]

I totally agree, and thats why I don't consider it a cheat. It is, however, deceitful, and that is what I don't like. Its an insult.

But I would like to know if the FX series also uses ps 1.4/2.0 in 3dmark 2001. They certainly could. If they are not then, if they did they would really kick the Radeons ass (even w/o cheating).
But I doubt that they aren't.


"Every Day is the Right Day." -Pink Floyd

 

[GeneticWeapon]

Will, Nvidia cards cant do PS 1.4, only ATi and Matrox.

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>120% Myself!</font color=red>

 

[GeneticWeapon]

im NOT trying to be an ATI fan

but, if the hardware has the capability to do something exactly the same with less work (ps1.4 vs ps1 or 1.1) then why not ?

Phial, I think it would be a great idea, but GT4 is calling specifically for PS 1.0, not 1.4, or 2.0. If ATi could implicate this type of shader swapping <b> in games </b> it would be great, but they dont. They are doing it only to achieve a higher 3DMark.

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>120% Myself!</font color=red>

 

[phial]

good point.

*hides in the corner again*

dammit

-------

<A HREF="http://www.quake3world.com/ubb/Forum1/HTML/001355.html" target="_new">*I hate thug gangstas*</A>

 

[GeneticWeapon]

Dont do that, I've missed talking with you.

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>Folding for Beyond 3D</font color=red>

 

[TheGreatGrapeApe]

The funny thing (to me) is I think you are both right. I think it's more of an optimization that a cheat if it were wide-spread. The idea is right, but it's implementation is faulty, thus tainting the whole procedure.
If the cards automatically forced the PS1.4 whenever possible fine, the fact that that only happens in 3Dmak01 makes it 'cheating', but I would say to a lesser degree than other examples we have seen, since as Phial says, there is no loss of IQ and also this would be reproduceable in actual games (unlike predicting the test path).
However that being said it's another example of the potential death of any reliable method of comparison outside of sitting down with the cards side by side in front of YOU personally and playing the crap out of themfo long periods to get a feel for them. Ticks me off!

I think the 'trusted' benchmarking is what truely pushes the companies, without it who's to say who's better. Well maybe image quality will make a return to supremacy over FPS, but we'll see.

Man I hate that EITHER company cheated.

It's time's like this when I wish Creative WAS in the fray with the P10, just to be able to be indignant about all this. Perhaps this is why ATI was fairly quiet about Nvidia's cheats. Hum Hum, yeah well, Hum hum, uhhh, we have no comment.



- You need a licence to buy a gun, but they'll sell anyone a stamp <i>(or internet account)</i> ! <font color=green>RED</font color=green> <font color=red>GREEN</font color=red> :tongue: GA to SK

 

[GeneticWeapon]

It's time's like this when I wish Creative WAS in the fray with the P10

If they were, I would need <b>you</b> to buy it for me!

Perhaps this is why ATI was fairly quiet about Nvidia's cheats

It was ATi who blew the whistle on Nvidia for the 3DMurk cheats

3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>Folding for Beyond 3D</font color=red>

 

[phial]

aweeeeeeeeeeeeee GW stop it! your maknig me blush across my chest .. see ? *shows GW*


teeheee....

ROFL *cough* anyhow



i thnk the questoin now is:

What is an optimization?

-------

<A HREF="http://www.quake3world.com/ubb/Forum1/HTML/001355.html" target="_new">*I hate thug gangstas*</A>

 

[Ghostdog]

When will it end? And how can we trust benchmarks anymore

<font color=red>I´m starting to feel like a real computer consultant.</font color=red>

 

[daddywags214]

Sucks for nVidia then if they can't do PS 1.4. That's really not cheating, man. THAT is a legitimate optimization that will improve realworld performance at no loss to IQ. That's like saying you can't add a turbocharger to a slow car because it wasn't included in the original design. Either way, that's not cheating. People have WAY overused the word "cheating" lately. Hell, half the stuff (and ONLY half) that nVidia's been doing hasn't been sheating IMO.

I don't think that optimization should be removed, although I thank you for bringing it to my attention. And by the way, how do you know that it isn't being implemented in the real world? Are there even any games to which that's applicable? Because the way I see it, if they can boost their 3dmarks/fps/WHATEVER without any impact on IQ or stability then I think it's okay. And that means ATi, nVidia, or anybody.

Tit for tat, butter for fat, ATi's dog kicks nVidia's cat

(Maximum PC)<P ID="edit"><FONT SIZE=-1><EM>Edited by daddywags214 on 06/16/03 08:33 AM.</EM></FONT></P>

 

[phial]

why do in 3 passes what can be done in 1?



-------

<A HREF="http://www.quake3world.com/ubb/Forum1/HTML/001355.html" target="_new">*I hate thug gangstas*</A>

 

[coolsquirtle]

WAHHAHAHAHHA ATi CHEATING TOO eat that ATi fanboys (just realise i'm being a noob) i mean urg.... bad Ati........

ANYWAYS
this news makes me happy and sad

happy- now ATi fanboys can't fart around going, "nVidia is cheap and crappy, ATi is right and good"
sad- sigh..... when is this ever gonna end.....

LIKE I SAID BEFORE
it's really all a conspiracy from both companies to trick us consumers. BOYCOTT NVIDIA AND ATI!!! LONG LIVE MAXTROX AND INTEL EXTREME GRAPHICS 2!!!!!

Proud Owner the Block Heater
120% Nvidia Fanboy
Athlon 64......nForce 3, it sure sucks when you can't afford things. One day Operton Servers will cost less than $4000 so everyone can get one

 

[GeneticWeapon]

why do in 3 passes what can be done in 1?

Because Futuremark wants you to do it in 3 passes.



3DMark 03 = 4,140
<A HREF="http://service.futuremark.com/compare?2k3=897633" target="_new">http://service.futuremark.com/compare?2k3=897633</A>
<font color=red>AthlonXP 2100+/Radeon 9500Pro</font color=red>
<font color=red>Folding for Beyond 3D</font color=red>

 

[davepermen]

Because Futuremark wants you to do it in 3 passes.

thats no reason.

a real reason:

because they cannot detect it automagically in _every_ app to optimize in _every_ app the same way. if they could, it would not be notable. it would be an optimisation. not a cheat.

the detectionmechanism for the following optimisation, THAT is a cheat. detecting "ah, its 3dmark, we can do this and that", _THAT_ is not allowed. the optimisation itself is. according to all specs and documents, they _are_ allowed to do that.

the only cheat is the detection on how-to-do it.

nvidia did real cheats. as well, combined with the faulty detection-mechanisms.

"take a look around" - limp bizkit

www.google.com

 

[rubikian]

Both cheating...only that Nvidia is a level above ATI...haha

 

[endyen]

Bothe cheating, AWfukit. I'm going to go play with my brand new built by Ati 9600 pro and pretend that none of this crap happened. You know why? I can still recomend the 9600 pro cause I know it's one hellova card.