Benchmark for Tom

[Guest]

Here is a email to tom and his response . These is a honest test that I did and I get different results from Tom on the performance beteen intel and amd cpu . This was snet to Tom and he did not seem to like it too much . See for yourself

PS. If you would like an exe and source code for the benchmark email me at yvonbou@videotron.ca

Hello Tom

Here are the test results on my system


Pentium II 450 MHz, ASUS P2B-F

Execution time: 68.739723 ms (nPixel: 1048576)
Execution time: 29.851272 ms (nPixel: 524288)
Execution time: 14.981788 ms (nPixel: 262144)
Execution time: 7.431389 ms (nPixel: 131072)
Execution time: 3.676723 ms (nPixel: 65536)
Execution time: 1.711390 ms (nPixel: 32768)
Execution time: 0.496152 ms (nPixel: 16384)
Execution time: 0.257295 ms (nPixel: 8192)

Athlon Thunderbird 1 GHz, ASUS A7V motherboard

Execution time: 87.334843 ms (nPixel: 1048576)
Execution time: 41.203840 ms (nPixel: 524288)
Execution time: 20.452879 ms (nPixel: 262144)
Execution time: 10.227277 ms (nPixel: 131072)
Execution time: 5.110985 ms (nPixel: 65536)
Execution time: 2.562616 ms (nPixel: 32768)
Execution time: 1.129473 ms (nPixel: 16384)
Execution time: 0.372394 ms (nPixel: 8192)


Here is the test software exe and source code . This allocates a buffer and does a mmx funtion between 4 source buffers and writes the result to a destination buffer. The function does image lookup function then a recursive average .



Yvon Bouchard
Coreco application manager

----- Original Message -----
From: Thomas Pabst
To: yvon bouchard
Sent: Wednesday, January 31, 2001 4:32 PM
Subject: RE: Your Benchmarks results


How can my test show different results from yours if you are actually using proprietary benchmarking software that is unavailable to me? A different benchmark gives different results and in fact I am trying to stick to real world benchmarks as much as I can.

I don't apprecaite your slight implication that my testing would not be reliable. What claims to fame in terms of benchmarking have you got ?

Regards,

Tom
-----Original Message-----
From: yvon bouchard [mailto:yvonbou@videotron.ca]
Sent: Thursday, February 01, 2001 1:59
To: tom@tomshardware.com
Subject: Your Benchmarks results


Hello Tom

I am an avid reader of your reports. I work in the image processing field and use embedded pentiums. We recently looked athlon 1 Ghz AMD and ran our image processing code . On raw memory access the AMD with the VIA 100mhz FSB required twice as many CPU clock cycles as the INTEL 933mhz 815 system. The MMX image processing functions also require 50 % more time on the AMD . I do not favor one or the other but my tests show a different result from yours . If you are interested I will email executables and source code of these simple tests . I am curious to why I see this discepency between what you note and what I can measure .

Yvon

 

[Guest]

Wow.. Tom is a little defensive....

 

[Guest]

I do not like to give my email on line but found no other way to attach a file for everyone to download. I have code and source code to demo what I say

Yvon

 

[Guest]

strange ... i did not detect any implications of Tom's tests being unreliable from your note ... you were just asking for the technical reasons that would cause this discrepancy.

one would have expected a response such as "thx for the info ... we'll have a look at it ..."

 

[Oni]

marf

 

[Guest]

Nah.. it's ok. If you are daring enough to post it.. then it probably has a ring of truth.

Truth is.. I don't do benchtesting anymore... The systems are all so fast as it is. The only benchtest I do now, is to see how fast those cards actually really disappear in the Free Cell game.

I have noticed that on my old 486.. it was slow. I could actually see each of the cards go up.

On my pentium, it was rather enjoyable to watch. Much like seeing a deck of cards being "shuffled".

On a celeron 466.. it is pretty damn fast. You can't see the cards individually, however, I can still spot all the cards disappearing.

P3-666 - DAMN FAST. One moment the cards are out, and the next moment, they are all in their neatly stacked piles.

This is about the only benchtesting I do now...

Jeff

 

[Guest]

Benchmarking what a wonderful thing, i also work with alot of Real time systems, for one project we also made a benchmarking floppy disk using a RTOS (QNX) that did a number of tests (simple integer calculation, Floating point and some image filtering algorithm's). On all tests except some that used mmx instructions my P200 mmx was able to beat a PII 400 and come close to a PIII 600. We also found that a PIII 600 could beat a PIII 800, unreliable benchmark is the obvious first thought however it was down to different motherboards switch the 800 CPU over to the motherboard where the 600 was and the 800 was wining by the expected margin. The code was then optimised for a PIII and my rather old box was unable to compete with the PIII 600

My point?
Well as Tom implied (should have said) proprietary benchmarking software could be optimised for anything (and it is normally in the favour of intel (Wintel)) also as tom has stated a number of times intel is better (clock for clock comparison) at 3D stuff this is mainly due to the mmx implementation which by the sounds of thing your tests use a fair bit. Therefore your test is not completly fair, i wont go into the test we did with an athlon but lets just say i would agree with a few of you result however floating point stuff is an other story

 

[Guest]

The problem that I saw shows up with a simple memcpy with buffers of 1 mb . The amd takes twice the amount of time the intel does . It turns out the memcpy is the worst case bandwidth to system memory . I have done a series of more test which I will post later which show this . Memory access is critical for the performance. The amd seems to suffer from that . In fact I suspect thats why DDR mother boards don't look that much better over sdram .

Yvon

 

[Guest]

P3's are slow? By what standard? Honestly, P3's are still pretty fast CPU's by any standard. The P4 has yet to prove that it's really any faster than the P3. And the T-bird is always a selective case. T-birds are faster in some apps and P3's are faster in others. Truthfully, both are good processors and P3's are still pretty fast. 667MHz? Well, you get the multiplier and the FSB and figure out for yourself who's really right about the speed of that particular P3 (here's a hint, FSB either 66, 100 or 133MHz FSB).

Basically, before you go and mock somebody you should do some research so that you don't end up making yourself look bad.

Charles

 

[TheSandman]

I still want to see a dirct comparison of the p3 and athlon on the via chipset lets isolate the cpu's as much as posible and go from the (I dont know of any reviews like this, so if you all do let me know).

SANDMAN

 

[pvsurfer]

We should all (Tom included) keep an open mind and not flame yvonbou over his benchmark and preliminary results. Without examining the program, it seems that its basis is sound and I for one would very much like to pursue this further.

 

[Guest]

Err... ok. 667 then.

Actually, to calculate it, you would use 133.33333333 * 5 which is 666.6666667 .. but we won't get into the mathematics about it. I am sure you were in line when they were handing out the mathematics lessons.

Slow CPU? P3 is a slow CPU? By what standards?

Perhaps slow to an Alpha chip, or slow to a Cray, or maybe even slow to the Motorola processors.

But on what accounts slow?

You keep running your AMD on that crappy VIA chipset. Speed you might win on, for SOME things, but reliability from Intel chipsets top.

 

[Guest]

Tom has not answered any of my following emails providing him code and exe's so he could explore the problem . Since he does not seem to be interested here is my investigation into this. Its a bit a sorry state of affairs to have to do this but somebody got to do it.

I have revised the tests and separated memory access from the MMX processing code . So the results below show three cases for each processors. One uses MMX and moves image data in and out of the processor . Next is only the MMX processing code on the data in the cache . Finally the memory copy function MEMCPY for different size buffers

Athlon 1 ghz vvia 100 FSB
Image processing performance
Execution time: 96.472317 ms (nPixel: 1048576)
Execution time: 43.765339 ms (nPixel: 524288)
Execution time: 22.005311 ms (nPixel: 262144)
Execution time: 10.954744 ms (nPixel: 131072)
Execution time: 5.497905 ms (nPixel: 65536)
Execution time: 2.738616 ms (nPixel: 32768)
Execution time: 1.046222 ms (nPixel: 16384)
Execution time: 0.366248 ms (nPixel: 8192)



MMX processing performance
Execution time: 86.603186 ms (nPixel: 1048576)
Execution time: 43.776513 ms (nPixel: 524288)
Execution time: 21.945527 ms (nPixel: 262144)
Execution time: 10.980166 ms (nPixel: 131072)
Execution time: 5.472204 ms (nPixel: 65536)
Execution time: 2.737219 ms (nPixel: 32768)
Execution time: 1.065219 ms (nPixel: 16384)
Execution time: 0.366248 ms (nPixel: 8192)



Copy performance
Execution time: 6.760636 ms (nPixel: 1048576) (155.100206 mb/s)
Execution time: 3.559112 ms (nPixel: 524288) (147.308673 mb/s)
Execution time: 1.761956 ms (nPixel: 262144) (148.780124 mb/s)
Execution time: 0.875810 ms (nPixel: 131072) (149.658093 mb/s)
Execution time: 0.069283 ms (nPixel: 65536) (945.923634 mb/s)
Execution time: 0.035200 ms (nPixel: 32768) (930.908973 mb/s)
Execution time: 0.008102 ms (nPixel: 16384) (2022.319492 mb/s)
Execution time: 0.005308 ms (nPixel: 8192) (1543.349086 mb/s)


P3 866 815 100 mhz FSB
Image processing performance
Execution time: 45.381174 ms (nPixel: 1048576)
Execution time: 20.423540 ms (nPixel: 524288)
Execution time: 10.093179 ms (nPixel: 262144)
Execution time: 5.022704 ms (nPixel: 131072)
Execution time: 2.655923 ms (nPixel: 65536)
Execution time: 1.266362 ms (nPixel: 32768)
Execution time: 0.611809 ms (nPixel: 16384)
Execution time: 0.096381 ms (nPixel: 8192)



MMX processing performance
Execution time: 39.813708 ms (nPixel: 1048576)
Execution time: 20.059806 ms (nPixel: 524288)
Execution time: 9.996798 ms (nPixel: 262144)
Execution time: 5.008456 ms (nPixel: 131072)
Execution time: 2.526019 ms (nPixel: 65536)
Execution time: 1.264686 ms (nPixel: 32768)
Execution time: 0.610133 ms (nPixel: 16384)
Execution time: 0.096381 ms (nPixel: 8192)



Copy performance
Execution time: 5.647923 ms (nPixel: 1048576) (185.656924 mb/s)
Execution time: 3.028038 ms (nPixel: 524288) (173.144479 mb/s)
Execution time: 1.437333 ms (nPixel: 262144) (182.382217 mb/s)
Execution time: 0.610133 ms (nPixel: 131072) (214.825208 mb/s)
Execution time: 0.254781 ms (nPixel: 65536) (257.224920 mb/s)
Execution time: 0.051962 ms (nPixel: 32768) (630.615932 mb/s)
Execution time: 0.013410 ms (nPixel: 16384) (1221.818368 mb/s)
Execution time: 0.008381 ms (nPixel: 8192) (977.454694 mb/s)


Finally my old over clock celeron 450 bx

Image processing performance
Execution time: 60.827267 ms (nPixel: 1048576)
Execution time: 26.151920 ms (nPixel: 524288)
Execution time: 13.010588 ms (nPixel: 262144)
Execution time: 6.557256 ms (nPixel: 131072)
Execution time: 3.252647 ms (nPixel: 65536)
Execution time: 1.616685 ms (nPixel: 32768)
Execution time: 0.798705 ms (nPixel: 16384)
Execution time: 0.380495 ms (nPixel: 8192)



MMX processing performance
Execution time: 52.411116 ms (nPixel: 1048576)
Execution time: 25.977596 ms (nPixel: 524288)
Execution time: 13.234360 ms (nPixel: 262144)
Execution time: 6.513675 ms (nPixel: 131072)
Execution time: 3.307961 ms (nPixel: 65536)
Execution time: 1.603276 ms (nPixel: 32768)
Execution time: 0.800381 ms (nPixel: 16384)
Execution time: 0.379657 ms (nPixel: 8192)



Copy performance
Execution time: 5.866666 ms (nPixel: 1048576) (178.734573 mb/s)
Execution time: 3.000380 ms (nPixel: 524288) (174.740504 mb/s)
Execution time: 1.501866 ms (nPixel: 262144) (174.545481 mb/s)
Execution time: 0.769371 ms (nPixel: 131072) (170.362474 mb/s)
Execution time: 0.366248 ms (nPixel: 65536) (178.939074 mb/s)
Execution time: 0.113143 ms (nPixel: 32768) (289.616206 mb/s)
Execution time: 0.032686 ms (nPixel: 16384) (501.258818 mb/s)
Execution time: 0.018438 ms (nPixel: 8192) (444.297588 mb/s)


So the raw memory access speed is about the same . The cache data accessing speed is very fast on the athlon. The surprise is the poor performance of the athelon on the MMX code .


I wish Tom would be doing this !!!!!


Yvon

 

[pvsurfer]

For whatever reason, you have butted heads with Tom, so why not make your case with some of the other hardware test-sites (e.g., AnandTech, Sharky Extreme, Tweak Town, X-bit Labs, etc.)?

As I said before, I for one am very interested!

 

[Guest]

Since Tom reacted this way is what that makes it intresting

Yvon

 

[IntelConvert]

I'm sure Tom and his staff put up with a lot of criticism, so you probably struck a sore spot (and at a bad time)!

Get over it (and I second pvsurfer's suggestion)...

 

[Guest]

I say we all get together for lunch an talk about it like civilised people. We can meet somewhere near Toms office, say noon Friday, at the Kentucky Fried Chichen.

Take Care.
Yes,I know I'm immature, but my goal is insanity!

 

[Guest]

I am also interested. And I also recommend talking to another hardware site if Tom isn't interested in this information. In addition to the ones mentioned you might want to consider [H]ardOCP.

John Garrett
System Admin - www.elementk.com
Editor - Exploring Windows NT Professional