Are Tom's Reviews Biased?

G

Guest

Guest
I have read Tom's reviews for quite some while, and I've always noticed something. And Underlying bias of sorts. A /very/ Pro-AMD, Anti-Intel feel which makes for sceptical reading. Before we get a swarm of "AMD ownz j00" spam, let's keep this away from which chip is better, and more towards "Does Tom represent his benchmarks fairly?"

Let's look at one of his more recent reviews found at: http://www6.tomshardware.com/cpu/02q1/020107/index.html or known as "The Eternal Race: P4 vs. Athlon XP". This is a review mainly looking at the difference between the AMD XP 2000 vs the P4 2.2 (Northwood).

Let's look at the benchmarks he provides:

At page http://www6.tomshardware.com/cpu/02q1/020107/p42200-12.html we see two benchmarks. On the Top Graph, the AMD XP 2000 scores 10460 and the P4 2.2Ghz a 10231 in 3D Mark 2000 test. The XP 2000 runs apparently at 101.35% the speed of the P4. Under this graph we see the text "... the AMD Athlon XP is able to dominate the scene and take the lead". Interestingly enough, in the bottom graph showing 3D Mark 2001 scores we see a similar comparison; this time the P4 with a score of 7452 and the XP with 7348. The P4 apparently running at 101.41% of the speed of the AMD. An unbaised report should, in theory, show at least somewhat similar text under this graph this time labeling the P4 as a "Dominator", no? After all, it did win by a slighltly greater gap, right? Well, what we see is "... the Pentium 4/2200 overtakes the lead by a nose ...". This is the sort of bias I am speaking of, though in all fairness it could be entirely the subconsious of the reviewer. Let's continue, shall we?

You can see for yourself through the rest of the benchmarks a sort of "excitement" any time the XP wins out, but very little is said when the P4 kills the competition. More specically, check out the NewTek Lightwabe 7b benchmark (http://www6.tomshardware.com/cpu/02q1/020107/p42200-15.html) and see for yourself.

As far as benchmarks, I think I've made my point, so let's skip to the conclusion, eh? (http://www6.tomshardware.com/cpu/02q1/020107/p42200-19.html)

Things like the below tick me off. Tom's page continually professes that they will not be taken in by the difference in clock speeds as they obviously mean /nothing/ when it comes to performance. Yet they say this as if in defense for AMD:

"After all, the top AMD processor has to make do with 1666 MHz, while its archenemy steps in with 2200 MHz. A closer look at the comprehensive benchmarks reveals that in Office performance as well as Linux Kernel compiling, the Athlon XP still takes the lead, despite its 32% clock speed disadvantage!"

I mean COME ON. The 32% drop in clock speed isn't a disadvantage at all, in fact it's what AMD prides itself over. A supposedly more efficient design. When an XP 2000 at 1.67 Ghz beats out a P4 1.9 Ghz, it's proof enough that clock speed is irrelevant when comparing AMD to Intel. They only reason they even appear is really for internal comparison, the 1.9Ghz P4 vs the 2Ghz P4. Don't try to use AMD's strengths as their weakness.

In the end, let me add - I'm usually an AMD supporter. If you don't overclock (like I do), and speed is your thing, AMD is a better buy then Intel most of the time. But when the new P4A beats the AMD XP's best in 13/20 benchmarks, clearly reigning supreme, why don't we see 3 cheers for Intel? An unbaised reviewer would, and I sure do.

Let the flames begin?

Another Note: Even though the P4 2.2 ghz is almost twice the price of the XP 2000, I hear you can OC to 2.57 Ghz w/o extra cooling. As I said, I'm an avid overclocker, and if you are too you might want to consider one when the price drops a bit (which they always do). Intel's are usually a better OC'ing buy, but that's another post entirely.
 
This has probably been discussed before. I will just say that I agree with you. Because of the nature of this thread, and the probable flames that will accompany it, I will just leave it at that.

-Raystonn


= The views stated herein are my personal views, and not necessarily the views of my employer. =
 
No flames from me. I am pro AMD but I have noticed biased in the benchmark comments. Most of the time pro-AMD, sometimes pro-intel.
Actually I am beginning to dis-trust Toms benchmarks altogether. He gets really bizzare results sometimes and doesnt bother to explain them.
Anyway, for the sake of reporting. Adjectives should be kept out of benchmark results. Just show them and let us decide what to make of them (unless something really wierd pops up).

Benchmarks are like sex, everybody loves doing it, everybody thinks they are good at it.
 
I am completely neutral when it comes to processors and I would say that you should simply ignore the comments made and concentrate only on the graphs. The comments are completely irrelevant.

AMD technology + Intel technology = Intel/AMD Pentathlon IV; the <b>ULTIMATE</b> PC processor
 
yaaa
this has been covered before... a couple of the reviewers at toms like using 'emotive' language... once there was a 1 second difference between an amd & intel cpu... something like 178 seconds compared to 179, and given that it was measured in seconds, not miliseconds, that could well have been due to error for all we know, yet somehow it got described as cpu A crushing cpu B

i do read toms reviews, yet i also go to other locations like hardocp or anandtech for reviews

"I came, I saw, I overclocked", Julius 'Smokin CPU' Caesar :smile:
 
P.S. if your an avid overclocker, get the 1.6A or 1.8A instead... you will achieve a much more rewarding overclock and have to pay far less to do so.

"I came, I saw, I overclocked", Julius 'Smokin CPU' Caesar :smile:
 
some ppl say pro-amd and some say pro-intel
whichever way it is its such a small bias it never really matters. I agree with AMD man, the comments are mostly irelevant neway


Trying is the first step to failure
 
We just make an informed decision and build what we can afford.
Best buy for the buck.

:smile: <font color=blue>You get what you pay for.... All advice here is free.</font color=blue> :smile:
 
YOu may pooobah, but alot of 1.6a's are topping out at 2.2ghz, you can get an axp 1700+ and run it at 2100+ speeds for less, which would give you more performance.

"The Cash Left In My Pocket,The BEST Benchmark"
No Overclock+stock hsf=GOOD!
 
While I have noticed Tom Pabst turning pro-AMD especially after all of the flak he caught from Intel on his RDRAM (i820) and P3 1133 exposés. I would too if I were under that kind of fire.

But, Frank Völkel in his latest articles (without the Tom byline) is pulling the same things pro-Intel, especially today's <A HREF="http://www.tomshardware.com/cpu/02q1/020225/index.html" target="_new">article on the P4 533MHz bus</A>. It seems obvious to me that to be fair he would have had to put the 133MHz P4s up against 166MHz Athlons. This would be easy through additional overclocking and 166 FSB Tbreds have been hinted at (but not officially) by AMD.

In addition, he consistenlty labels RDRAM as fast and DDR as slow. While his table indicates the reality; shipping DDR sticks are faster than shipping and upcoming RDRAM, he hardly makes passing mention of the fact that RDRAM platforms for P4 perform better only because they have the benefit of dual-channel.

My point? Well, Tom's can be infuriating when your team doesn't win the latest review praise. Sometimes it can even make you change teams - score one for truth. Overall, I think Tom's is the best review site on the net, but I still want a second, third, forth, fifth...opinion.

I thought a thought, but the thought I thought wasn't the thought I thought I had thought.
 
he hardly makes passing mention of the fact that RDRAM platforms for P4 perform better only because they have the benefit of dual-channel.
Right, because it's not like RDRAM is actually at a disadvantage, it being 32-bit and SDRAM being 64-bit.

Er...hold on a minute...


As for Tom's/Frank's/Bert's/Ernie's/Bubba's/whoever's reviews, I say ignore the comments and look at the benchmarks. For one thing, there has to be a lot lost in the translation. (As others have said) read the benchmarks, ignore the comments.

Also, there have been hordes of people claiming Tom is biased towards not only Intel, but AMD, Ati, nVidia, Via, SiS, Ali, Maxtor, IBM and the Chicago Cubs. I'd say that if people complain both ways, it's a good indication that the only bias is review-to-review. It's hard to NOT be biased in a review when you really like the product personally. For example, if Intel released a 5.0GHz P4 tomorrow, and it was reviewed here on THG, then people would scream that he was biased towards Intel because he went on and on about how great it performed. If AMD released a 5.0GHz AXP (which would be quite a feat), and THG reviewed and praised it, then people would scream that he was biased towards AMD. Make sense?

<font color=orange>Quarter</font color=orange> <font color=blue>Pounder</font color=blue> <font color=orange>Inside</font color=orange><P ID="edit"><FONT SIZE=-1><EM>Edited by FatBurger on 02/25/02 08:31 AM.</EM></FONT></P>
 
Right, because it's not like RDRAM is actually at a disadvantage, it being 32-bit and SDRAM being 128-bit.
I don't quite get what you're driving at. If you are being sarcastic, I would agree that RDRAM should not be at a disadvantage due soley to its 16bit/32bit dual-channel nature; it should be able to ramp to higher bandwidth quicker than DDR, but the fact remains that it hasn't. I don't think it will. Check out <A HREF="http://www.anandtech.com/chipsets/showdoc.html?i=1588" target="_new">Anand's latest article</A> on the new E7500 dual-channel DDR chipset for the P4 Xeon proc. If RDRAM were better and faster, why would Intel pick DDR for its mission critical servers? Maybe the wool will finally be lifted?

Oh, and I agree about the rest (except that quite a feat biased part...It would only have to run at 3400 to rip that 5GHz P4 to shreds - and will - the clawhammer). :lol:

Flame war here we come...who will be the first to call the other moron, idiot, etc.?

I thought a thought, but the thought I thought wasn't the thought I thought I had thought.<P ID="edit"><FONT SIZE=-1><EM>Edited by ath0mps0 on 02/25/02 10:43 AM.</EM></FONT></P>
 
RDRAM probably hasn't ramped up because everyone at Rambus is idiots :tongue:

Haven't seen that article at Anandtech, I'll give it a quick look.
Mmm....dual gigabit ethernet 😎
Very unusual board layout. You don't see too many dually boards with the two sockets squared off where DIMMs/RIMMs usually are.
Question for whoever can anser: what's the little "extension" on the first 64-bit PCI slot?

Anand pointed out the possibility that this board is why nVidia doesn't have a P4 license (Intel holding off until they can get this to desktop).
That board is amazing though, just jammed with controllers, resisters, capacitors, etc. No free space whatsoever.

As for dual-channel DDR, there could be any number of reasons that Intel is using that instead of RDRAM. Maybe Rambus wants to do things only their way, and won't let Intel make requests for what platforms they want. Maybe Intel and Rambus had a falling out. Maybe Intel just thinks DDR-SDRAM would be better (I'm not saying that's not the case). Who knows. I'll read through the article properly later on.

<font color=orange>Quarter</font color=orange> <font color=blue>Pounder</font color=blue> <font color=orange>Inside</font color=orange>
 
I just want to add my 2 cents.

From my observations, the comments (and often times the conclusions as well) in THG reviews are often biased towards the underdog, just as sheps observed. Even at the best of times there still always seems to be a biased undertone.

Granted, it may be subconscious. It may not be intentional. THG staff may not even be aware of it. However, myself and plenty of other readers have noticed it, so there must be some truth to it.

In my not-so-humble opinion, personal opinions and biases don't belong in reviews. I see it as a matter of professionalism. (Something which THG has at times lacked.)

ath0mps0, I have to say that your comment about to be fair the review would have compared the new P4s to a 166MHz bus Athlon, well, comical. Has THG recieved an engineering sample from AMD set to a 166MHz bus? THG reviwed an AMD platform using the KT333 chipset and PC2700 memory. All considered, it was the best they could offer. THG doesn't compare vaporware to actual hardware.

Further, your statement of, "In addition, he consistenlty labels RDRAM as fast and DDR as slow.", has no more forethought than the previous comment. His conclusions are clearly drawn for two reasons based on comments in the review. 1) The performance gain seen from new RDRAM systems over new DDR SDRAM systems is quite evident. (While I admit that making such a conclusion is a bit premature given that THG has not successfully run PC2700 at CAS2, it is at least based on what hardware is currently capable of, which is all that anyone can ask. Anything else is just theory.) 2) The P4 clearly performs better when running RDRAM. This is due to synchonous clocks and bandwidth available through chipset/motherboard support.

Then ath0mps0, you continue to show an unusual penchance for tunnel-vision by asking, "If RDRAM were better and faster, why would Intel pick DDR for its mission critical servers", after poiting out a review on a new Xeon chipset. The simple answer is that Intel started work on an RDRAM solution FIRST and DDR second. Besides, just because something is new doesn't mean that it is the only or the best.

FatBurger, you make a good point that there will always be those who consider THG biased just because they review the best-of-the-best as being the best and these people can't accept that their side has just lost face.

However, even when THG reviews Intel as the new performance leader, there is still an air of AMD favoratism. Yet when THG reviews AMD as the new performance leader, AMD 'crushes' Intel. It seems to me that it there were no overall bias, then clear-cut winners would ALWAYS be declaired as such. No more, no less.

<pre><b><font color=orange>AROOOOOOOOOOOOOOOOOOOOOOO!</font color=orange></b></pre><p>
 
(Haven't seen you around in a while, BTW. How've you been?)

However, even when THG reviews Intel as the new performance leader, there is still an air of AMD favoratism.

True, but the fact still remains that many people accuse THG of favoring Intel. So the question is, if THG always favors AMD, then why are there people saying the opposite?

<font color=orange>Quarter</font color=orange> <font color=blue>Pounder</font color=blue> <font color=orange>Inside</font color=orange>
 
Heh heh. LOL @ FatBurger.

Yes everyone at Rambus does appear to be an idiot.

A more truthful statement though would be simply how many hours and dollars have gone into DDR SDRAM research and development compared to how many have gone into RDRAM r and d. RDRAM hasn't ramped up because nowhere near the resources are going into it compared to DDR SDRAM. (In fact, for as much as goes into DDR SDRAM, you would think that we should have quad-rate x 200MHz modules by now.)

<pre><b><font color=orange>AROOOOOOOOOOOOOOOOOOOOOOO!</font color=orange></b></pre><p>
 
Oh, I've been busy like a millionare madman on crack in a whorehouse. Funny thing how life can get busy when programmers keep leaving the company, leaving their workload on the rest of us. Sometimes though, you just gotta slow down and take a break before you start doing more harm than good to your code.

Seriously though, you already answered your own question of, "if THG always favors AMD, then why are there people saying the opposite?". The answer is individual bias of the readers. They can't stand to hear that someone else's latest-and-greatest beat out their one-and-only true love. So instead of admitting that Company-X produced a better product, they claim the review to be biased. That way they can continue to live in their own little dream world without being disturbed by reality. :)

Speaking of disturbed, I think I've probably spent more of my lunch break than I should have already. It's time to jump back into the madness and whip these 1s and 0s into shape.

PS: I'd consider wondering where the sanity lies in making my life's work revolve entirely around adjusting magnetic and electrical impulses like so many bits of flotsam in the ocean if I weren't afraid of the answers that I'd find.

<pre><b><font color=orange>AROOOOOOOOOOOOOOOOOOOOOOO!</font color=orange></b></pre><p>
 
I wonder.....
You say its insane to think a 166mhz bus XP should be compared to the new 133/533 intel, and you are partially correct. However you also mention that there is no vaporware review, but there is! Intels 133/533 setup wont be available for 2 more quarters! By that time there will be new AMD chips and setups so I dont think this article relly means much yet.

As for RDRAM, I dont think there is a real comparison between it and DDR cuz there is no dual channel DDR for P4. There is also no RDRAM for AMD. If there was either we could see a true comparison, but as it is we are comparing apples to oranges.

I myself think that each person should see what they are doing on the machine and look at benches of those apps. Once you know what you need just buy the best value. I does not matter if you "only" need a 1ghz pc or the fastest available, the price/performance in what you use is the most important, esp keeping in mind that 90% of users wont notice the diff between a 2 ghz (or 2000+) CPU and a 1ghz in thier browsing and office apps =/

Jesus saves, but Mario scores!!!
 
Nice to see you around here slvr.....


I don't understand what the big fuss is. I don't even care what the reviewer thinks unless there is something really good, or really bad. I generally just look at the graphs and decide for myself. I then compare prices and the decision is generally easy to make. Biased or not, Tom's reviews have included a number of very interesting if not industry shattering results. The Pentium 1.13 comes to mind right away.

<font color=red>God</font color=red> <font color=blue>Bless</font color=blue> <font color=red>America!</font color=red>
 
I have to say that your comment about to be fair the review would have compared the new P4s to a 166MHz bus Athlon, well, comical. Has THG recieved an engineering sample from AMD set to a 166MHz bus? THG reviwed an AMD platform using the KT333 chipset and PC2700 memory. All considered, it was the best they could offer. THG doesn't compare vaporware to actual hardware.
I didn't find anywhere in the Tom's article that stated that they had received engineering samples of 133MHz P4s; in fact (apart from the dramatic license at the beginning) the voltage overclocking exposé led me to believe that they had just overclocked NWs and used PC1066 RDRAM. Oh, and Tom's does often benchmark overclocked CPU's To support his claims that "Our benchmark results clearly prove that if Intel changes the FSB and memory clocks (to 133 MHz and 533 MHz, respectively), this will put it quite a distance ahead of its competition from AMD, as well as its own series of processors." Frank should have applied this same logic and overclocked equivalent AXPs.

The performance gain seen from new RDRAM systems over new DDR SDRAM systems is quite evident.

snip

The P4 clearly performs better when running RDRAM. This is due to synchonous clocks and bandwidth available through chipset/motherboard support.
Correct. The reason I object to Frank's handling of this topic is that he consistently points to the type of RAM as the determining factor of performance while the actual root of performance is the chipset and synchronous clocks as you stated.

you continue to show an unusual penchance for tunnel-vision by asking, "If RDRAM were better and faster, why would Intel pick DDR for its mission critical servers", after poiting out a review on a new Xeon chipset. The simple answer is that Intel started work on an RDRAM solution FIRST and DDR second. Besides, just because something is new doesn't mean that it is the only or the best.
Actually what you call tunnel-vision is just being well informed and knowing some facts ahead of time; Intel let its channel partners know that they would not be supporting RDRAM on their server chipsets about the time the i840 was released. The two primary reasons were 1) Price of RDRAM - although now invalid, as anand states in his review of the chipset, 4-16GB would have been impossible for all but the largest organizations. 2) Memory Latency - with RDRAMs already high latency, it about doubles for every stick you add - it is a serial channel. They would have had to implement quad or octa-channel to overcome this issue that is an extreme factor in large active databases. IMO, they only stuck with RDRAM for the P4 i850 desktop chipset due to their contract with Rambus and to keep their hands on those, now practicaly worthless, Rambus stock warrants. In addition, they had done all of their development on the RDRAM platform and it would have delayed the lauch of the P4 - not that that would have hurt them too badly - the high cost and poor performance of early P4s was enough to convert many to Athlon.

FatBurger, you make a good point that there will always be those who consider THG biased just because they review the best-of-the-best as being the best and these people can't accept that their side has just lost face.
In MY not-so-humble opinion, I think that is what I said....

All in all, I only gave these examples (in my previous posts) to illustrate the contra-point to the initial article. Yes, Tom's Hardware often portrays a biased view from multiple perspectives - something that would be eradicated in a purely objective environment - the "perfect review site." I think that Tom's provides the closest thing I've found to this - but $.05 is closer to $1.00 than $.01....

I'll be the first to admit my bias for AMD; this has developed over time (from my first 80286 that had Intel and AMD stamped on the chip). After the first P-ratings failure I became a staunch Intel-only proponent. It was actually Intel's false and frankly, inept, handling of the i820 fiasco that drove me to AMD.

Now I am biding my time waiting for Intel to finally shrug off the Rambus monkey (the company) and use whatever technology is best for the consumer, not just best for the shareholder.

I thought a thought, but the thought I thought wasn't the thought I thought I had thought.
 
The answer is individual bias of the readers. They can't stand to hear that someone else's latest-and-greatest beat out their one-and-only true love. So instead of admitting that Company-X produced a better product, they claim the review to be biased. That way they can continue to live in their own little dream world without being disturbed by reality. :)

Couldn't have said it better myself.

should be compared to the new 133/533 intel, and you are partially correct. However you also mention that there is no vaporware review, but there is! Intels 133/533 setup wont be available for 2 more quarters!

However, the 133(533) P4 actually exists. If THG had a 166 Tbred in their hands, then they could review it and compare. The 133(533) isn't vaporware because it exists, doesn't matter if it's available publicly or not.


<font color=orange>Quarter</font color=orange> <font color=blue>Pounder</font color=blue> <font color=orange>Inside</font color=orange>
 
My AXP 1600+ @ 1750MHz w/166MHz FSB (~2133+) does exist!!! And it is fast.

I thought a thought, but the thought I thought wasn't the thought I thought I had thought.
 
You know it's a bad day when you spend two hours trying to figure out why your program suddenly stopped working only to finally realize that when working with miliseconds it helps to actually divide by 1000 or else you could be waiting for a REALLY long time...

That said, I need a break. :)

kief, you have part of a point. Intel's new hyped-bus wonder won't be available for a while. However, as the image posted in the review clearly shows, it's not vaporware. THG does have a legit engineering sample to play with. Hence it's more than just theory, and it's more than just a bunch of overclockers going nuts. It's a physical reality that someone can actually reach out and touch, and we can be sure that short of some catastrophic meteor-type end of the world, we'll see end-user production models sometime this year. However you are right in that until we can actually buy one, it really doesn't have much meaning to anyone other than the dreamers. :)

And the article that ath0mps0 linked to IS about an Intel dual-channel DDR motherboard, so before long we will probably have that age-old question answered as to which is better as Intel is finally taking DDR SDRAM seriously. Even then though, I expect that in the end we will find that even with equivalent systems that only vary by memory type, we will still find that they are indeed apples and oranges. :)

And I completely agree with you when you say, "I myself think that each person should see what they are doing on the machine and look at benches of those apps." I just also think that THG should try to be less biased in the meantime, because not everyone is as smart as us.

Some people do have little to no mind of their own. True, they probably deserve to be controlled by others then. However, true professionals shouldn't go around warping their minds with their own personal bias. The point of THG (so I thought) was to provide unbiased product reviews as a safehaven from marketting hype for consumers to find the truth. My opinion is that in this, THG has strayed.

dhlucke, it's nice to see you and FatBurger around still. It's nice to see people I know other people can trust still around. And you're right, THG has had a number of industry shattering reviews and articles. I've never said that THG is evil or bad. In fact, I find the site very useful. I just think that they are a tad biased, and that such bias degrades my opinion of their professionalism. That doesn't make them any less experts or their benchmarks any less informative. It just makes me less likely to recommend reading THG articles to other people who are trying to educate themselves in preperation for making an informed purchase. I still respect the THG staff. I just think that they could improve the site by leaving personal bias out of their reviews.

<pre><b><font color=orange>AROOOOOOOOOOOOOOOOOOOOOOO!</font color=orange></b></pre><p>
 
That's an overclocked chip, though. You can't really review overclocked products and say that they're the same as the retail version.

<font color=orange>Quarter</font color=orange> <font color=blue>Pounder</font color=blue> <font color=orange>Inside</font color=orange>
 
No one questions that. However, was it set to a 166MHz FSB at the factory? Can AMD produce enough of them at that FSB to actually launch a new line? Would AMD back up said 166MHZ FSB CPU as a part of their near (this fiscal year) future?

Or is yours just part of the rare minority of AMD CPUs actually capable of being overclocked so extremely and still run with stability?

That is the difference between vaporware and an engineering sample for running benchmarks/testings/reviews.

<pre><b><font color=orange>AROOOOOOOOOOOOOOOOOOOOOOO!</font color=orange></b></pre><p>