how is that unfair benchmarking? We use benchmarks to compare processors. We know amd is behind. Amd will catch up and possibly get ahead of intel. It happens all the time. Intel got ahead with its pentium 3. Amd got ahead with its athlonxp and 64. The cycle repeats itself and so on and so on. So how does this have anything to do with unfair when we are just comparing 2 different processors?
The problem is the "bennchmarks" used by the enthusiast press are garbage and subject to manipulation. ANSI/ASTM have written testing proceedures that all legitmate testers are required to follow. In 1988, the Standard Performance Evaluation Comittee (now Corporation)http://www.spec.org/spec/ was formed for the purpose of
"The goal of SPEC is to ensure that the marketplace has a fair and useful set of metrics to differentiate candidate systems."
What I find interesting and disgusting is that the enthusiast press when questioned about why they don't use these standards, which are the ISO9001 standards, excuse themselves by saying that that is not what their readers are interested in. What the enthusiast press is saying we aren't going to comply because doing so will eliminate our ablity to manipulate the results in favor of the person who paid us to run the comparasion.. The testing results you see posted across the enthusiast press, including here at Tom's, are what is commonly called "Junk Science". Not a single test in the enthusiast press is done in accordance with ACCEPTED testing proceedures. There is a legal definition of "accepted" that means following the ANSI/ASTM guidelines and the press benchmarks don't meet it. So don't quibble about what accepted means.
What you're asking review sites to use is more for business apps and companies, not for consumers.
I mean, SPEC might work for the person wanting to upgrade his/her workstation, but not for Joe Computer Builder who wants to build a kick ass gaming rig. I mean, why do I need to know how fast my computer can run Maya, when I just want to see F.E.A.R. run smoothly? Or do I need to know my system's CAD capablity, when I just want Oblivion to run good? And does this SPEC use everyday programs like LameMP3 or video encoding?
I don't doubt that there is a standardize benchmark, but those might be more for server environment, and not so much for desktops.[/quote
If you are going to proclaim that one processor is superior to another you are going to test all its functions not just a few of the easiest ones. The engineers and computer scientists have three standing commitees for 18 years that design the tests to test a cpu underall conditions that reflect use across the board from doing data conversions to calculation of differential equations to 3D graphics While they may not use the exact same benchmark as you quote LAMEMP3, you would know the harder the test the better the results carry over. We wouldn't have these lame situtations where testers are having to backtrack on their results because they either fudged the setup or used an inappropriate or obsolete version of a test. If you want to run Oblivion that is fine, but qualify it as a nonaccredited test, you can legitmately say one procesosr is superior to her in Oblivion. That one test is hardly the basis to proclaim superiority of one cpu over another. You did learn about the scientific method in high school I hope. Nonaccredited tests should be treated the same way nonaccredited diplomas are. The greatest benefit of the accredited tests is that they point out the falacies of the other tests. If you see major disparities in results then you probably rethink the validity of your conclusion. Your analogy using Obliviion or LAMEMP3 is similar to the thought process of Pope Gregory VI on why the earth is the center of the universe despite the observations of Copernicus and Gallileo. The other advantage to the standardized tests is that the testers cannot rig the results without a very high probablity of getting caught when submitting them for review. There are components in that series of tests that will test the 3D rendering and 3D phyics more than any game out there. The best example is the Gauss cluster at Lawrence Livermore which is use to display the output on a 4Mx5M display.
http://www.llnl.gov/PAO/news/news_releases/2005/NR-05-11-04.html
A single cluster from that unit in gaming would destroy anything anybody here can dream up. It was designed and built using the very tests that you claim are useless.
Facebook
Twitter
Instagram