Samsung Accused of Manipulating Benchmarks Again

Page 3 - Seeking answers? Join the Tom's Hardware community: where nearly two million members share solutions and discuss the latest tech.
Toggle sidebar Toggle sidebar
Status
Not open for further replies.
"It's not much different than the Surface RT having a 32gb SSD. Yes, in theory, absolutely true. When 14 or so gb of that is taken up prior to you even turning the device on for the first time, advertising 32gb of storage without a notable footnote is pretty disingenuous. Go figure, people were *flipping out* when MS misrepresented the storage space on their tablets like that..." Computers have the same thing since forever, and you don`t need a footnote for that ... why suddenly you need it now ? Everything in America needs a footnote/warning label these days... of course the damn OS will eat up some space, every device that has an OS has this " problem"
 


It's an issue of perception and relative percent of total space taken up. Taking up 10 to 20 gb of a 500gb hard drive is represents a loss of under 5%. The loss of space due to OS on the Surface was close to 50%. In actual working conditions, one is bordering on an insignificant percentile, while the other is VERY significant.

With the conventions of advertising storage space being what they are, it was playing pretty fast and loose. Heck, some of the claims about "it can hold X pictures and Y songs" which are very common for phones tablets counted the full 32 gb of storage for its calculations. And yes, we always expect the OS to take up some of the initial space, but while they were advertising the initial space on the SSD, they were not advertising that the OS took up several times as much space as their competitors' OS's.

What it all comes down to is that they advertised OS to make it sound like the basic model of the tablet had a lot of space on it, when in reality it had about half of what was advertised. With the capabilities of the tablet they were pushing, being a media center and whatnot, this is sketchy at best.

(Typed from my Surface RT which I'm really bloody happy with - but I'm not above admitting that MS's shady advertising here).
 
@iatemyelf
I read the article and understand it fine and as they stated they tested with the geekbench named normal & tested it by renaming it. All this shows is that Samsung is using performance profiling to make their hardware run at full speed when the hardware is running these benchmarks, It is not like they are forcing their hardware to run at faster than advertised speeds as in making the 2.3Ghz CPU run at 2.5Ghz. If they did that then I would cry foul play for sure.

As I said before if they forced their hardware to run at the max speeds all the time the battery life would be cut in half or worse so to me that would not make sense at all. So to those that complain this does not simulate what the device does in the real world of apps you should be happy that sammy did not force the hardware to run full bore all the time. I can not understand how some people complain that a device company is putting their created device in it's best light when running benchmarks.

Also has the site that is complaining about this also tested the device in real world apps? If not maybe they should to see how it performs in the top rated or newly created games if it runs everything with ease I fail to see what the problem is here. I do understand what some are trying to say here but at the same time I understand why samsung has done this as well. Back to the car thing as well if you bought a car & know it could do 0-60mph in 3.9 seconds does that mean you would drive it like that everyday of coarse not just because you could does not mean you would. Same thing with these devices they are getting very powerful for their size & as such do not need to be running full bore to get good performance out of them.

Like I said I see where people are coming from in all of this but I also see samsungs side & if other companies are complaining about it they are most likely upset that they did not think of it first & now their products are looking a bit slower even though they might have the same hardware in their devices. I can bet that in a short while we will be seeing some if not all companies doing this just like PC hardware makers have been doing for years.

This is a new trend that will catch on & I guess someone had to get the ball rolling for it & samsung seems to be that someone this time. Next year this time no one will notice or even care because everyone will be doing it. I personally think that only those that care at this point are probably so butt hurt that their new LG G2 or whatever phone just came out seems to be slower than the newly released note 3 so they are complaining just to complain about something. Just my 2 cents on this. I also am not directly attacking anyone on this I as I said everyone is entitled to their own opinion.
 
@hardrock40: it's about honest results, its not about my G2 runs lower benchmark results than your Note 3 even though we have the same processor.

And your car comment completely misses the entire point.

P.S. if it does 0-60 in 3.9 and you do accelerate fast when lights go Green Lights or other situations it may matter. Don't speak for everyone, I like to accelerate at lights and continue to catch the following lights, I don't care that it uses extra gas.

If it actually did 0-60 in 4.875 seconds but they unlocked something for a tester to make it go 20% faster to bring it down to 3.9 seconds and that's what's being told everywhere, damn right I'd be pissed. I could have bought another car maybe cheaper with the same 4.875 performance. Or waited and looked for a car that actually did 3.9 seconds for its customers not just for testers.
 
rwinches
Of course synthetic benchmarks are not pure indications of real world use.
The ones that are put together from real apps are better.
No one seems to mind when Toms uses apps that favor Intel designs in their reviews.
Samsung is trying to create MobileBench to assess hardware/firmware solutions for developers as well as reviewers, I think they have the right idea.
http://www.tomshardware.com/news/benchmark-consortium-mobilebench-mobilebench-ux-samsung,24419.html
Click to expand...

Fantastic, we'll have the company who manipulates benchmarks be a leader in setting mobile benchmarks.

Maybe we should make Enron execs. run the department of Energy...

Oh, if you're gonna say they have others on board, not interested in those Chinese companies with loose rules and regulations having a stake in how things should be standardized.
 
Just amazing how far Scamsung will go to beat fellow Android companies and Apple. Samsung does it fact make quality products but its business ethics are extremely poor. I liked Samsung until I read all about their business practices in Korea and abroad.
 
@wemakeourfuture
How is it not honest results All samsung did was make sure the hardware was running at peak output for a standard set of benchmarks that sites run they have to make a profile to do that & that is what they did. it is not like they forced the hardware to run at higher spec than what was listed on the spec sheet. Now if they did run the hardware at a higher spec than what was listed I would have a problem with that. because that would be considered cheating then. But they did not all the internals were running at advertised speed but it was running at peak output that the device could do. How is that cheating if the device is running a spec.

Just because LG or whoever did not think far enough ahead to do the same does not make it wrong. I also do not own a note 3 I own a Sony Xperia ZL wich is unlocked & rooted & overclocked by my choice & yes it does score higher than a standard Z or ZL.
mine runs now at 1.7Ghz for everyday use & 1.9Ghz for benches GPU runs now at 425Mhz & 450Mhz for benches & scores about the same as a S4 now or a bit higher sometimes. Now if Samsung was to do that to their device without disclosing it I would say they were cheating but that is not the case.
 
@wemakeourfuture
How is it not honest results All samsung did was make sure the hardware was running at peak output for a standard set of benchmarks that sites run they have to make a profile to do that & that is what they did. it is not like they forced the hardware to run at higher spec than what was listed on the spec sheet. Now if they did run the hardware at a higher spec than what was listed I would have a problem with that. because that would be considered cheating then. But they did not all the internals were running at advertised speed but it was running at peak output that the device could do. How is that cheating if the device is running a spec.

Just because LG or whoever did not think far enough ahead to do the same does not make it wrong. I also do not own a note 3 I own a Sony Xperia ZL wich is unlocked & rooted & overclocked by my choice & yes it does score higher than a standard Z or ZL.
mine runs now at 1.7Ghz for everyday use & 1.9Ghz for benches GPU runs now at 425Mhz & 450Mhz for benches & scores about the same as a S4 now or a bit higher sometimes. Now if Samsung was to do that to their device without disclosing it I would say they were cheating but that is not the case.
 
@hardrock40: do you still not get it?

They have SPECIFIC instructions for just benchmarks. Their architecture does NOT perform with regular apps with the same strain as it does with these benchmarks.

Let me give you another car analogy.
You know an automatic car, the gears change automatically (hope you know this, because all your car responses don't make any sense). They shift at various RPMs.

Imagine Car & Driver puts a key into a BMW 3 Series 323i. Its an automatic, but the car detects its for testing, so instead of shifting 1st gear normally at 3000 RPM it does it at 3600 RPM (20% difference), and as a result certain performance tests come up better.

Yes the car is capable of even going to 8000 RPM, it's physically possible, but what the end user will achieve with the car is different because their key does not unlock special mode in the car.

Do you get it now? It's not about the hardware being physically able to do it. Its about manipulating and inflating results specifically for benchmarking and cannot be replicated with other apps even if they requires similar resources.

If you still don't get it, nothing I can do further.
 
How is this cheating?


Also phones perform better (like laptops and tablets) when plugged in, since the cpu wont be in power savings mode as aggressive. So what's next, plugging in the phone they will say is cheating too?

 
AndroidIsKing
BOTTOM LINE:

ANDROID IS 80% OF MARKET SHARE

SAMSUNG MAKES BETTER PHONES THAN APPLE AND HAS A BIGGER MARKET SHARE.

SAMSUNG OFFERS MORE CHOICE.

ANDROID WON THE MOBILE WARS AND APPLE IS LIKE IN THE 90S
Click to expand...

1. Android does not have 80% of the Smartphone market which Apple only makes products of, 80% of phone market is one thing, smartphones is another. Also, the bulk of Androids are bought in developing world countries like China, India, Brazil and 2nd-tier countries like Russia. These are cheap crappy android phones not flagship phones like Galaxy, HTC One, Nexus.

2. Samsung making better phones is subjective. Samsung having a bigger marketshare, compare smartphones to Apple's and its not bigger. Also compare numbers of Apple to Samsung (not Android) in developed nations like (USA, Canada, Australia, Japan, Europe) and you see a completely different number than developing nations.

3. Samsung does offer more choice, so why try to cheat benchmarks?

4. If Android won, why is Apple selling 9 million smartphones in a launch weekend. Numbers any Android manufacturer would die for including Samsung? Why is Apple's market cap as a company bigger than (Samsung + HTC + LG + Sony + Motorola + ... +)?

Seems like they haven't won, when Apple can get +$100 billion in profits in a year more than most these companies are worth a few times over, seems like your definition of "won" isn't aligned with reality.

iPhones are the highest selling phones and electronic devices ever. No other electronic devices from a single product line has sold more than the iPhone in human history.

Compare any Samsung product sales to iPhones and it definitely is not a 1990s situation.
 
@AndroidIsKing

GM has a bigger marketshare than BMW or Mercedes, I guess they won too?

Apple has a the biggest marketshare in tablets, I guess they won that!?

Your arguments are stupid and childish. No person with a financial or technical background will give you the time of day with your silly points.

P.S. About your pre-sales numbers (again, nice try, Apple has the biggest numbers in history):

http://www.pocket-lint.com/news/121110-samsung-galaxy-s4-performing-better-than-galaxy-s3-to-surpass-10-million-sales-next-week

Unless you're gonna say Pre-orders were 9 mil for S3 and 3 weeks after that they only sold 1 million, which I can find another source to show how ridiculous that sounds.

Yes, Android has +90% of the cheapest phone markets, but when it comes to actual flagship phones its nowhere near that hence why they struggle with margins and profits except Samsung (which doesn't sell as many flagship phones as Apple need I remind you).

iPhone sales are unmatched in the industry, you can huff and puff all you want, numbers are well known.

And if you think marketshare is all that matters you don't understand business.

 


Plenty of people here that see what's wrong with this, but also have (and enjoy) Samsung hardware at the same time, without really caring what a benchmark says. I see discriminate use, not stupidity, stop calling other people names just because their preferences are not yours.
And no, you never thought Apple fanboys were bad (let alone say it). I have to call you out on that lie. All your posts have been pro-Apple and anti-Android, quit pretending you're impartial. You and otacon belong in the same Apple fanboy category. It is what it is.
 
house70
Plenty of people here that see what's wrong with this, but also have (and enjoy) Samsung hardware at the same time, without really caring what a benchmark says. I see discriminate use, not stupidity, stop calling other people names just because their preferences are not yours.
And no, you never thought Apple fanboys were bad (let alone say it). I have to call you out on that lie. All your posts have been pro-Apple and anti-Android, quit pretending you're impartial. You and otacon belong in the same Apple fanboy category. It is what it is.
Click to expand...

I'm not an Apple fanboy, LOL.
Just providing actual information.

Second it has nothing to do with 'preferences' different than mine. Its about all the comments where they clearly don't understand what the article is about and write something that is completely incorrect.

Everyone is entitled to their opinions, even if they are completely wrong and absurd.

P.S. I'm not anti-Android. I've owned Google stock and recommend Android devices to people who have a higher marginal utility for one. Others who I know would have a higher marginal utility for an iPhone I recommend that.

I actually look at things objectively and am not married to either camp, nor married to hating a side. I am not bound by dogma like many on TH.
 
It's not good Samsung unleashes the beast for the benchmarks but not for much else. It just isn't that important to me. The Note 3 is still irrevocably bad-azz. Samsung's "cheating" doesn't change that and wouldn't alter my purchase decisions in the slightest. The Note 3 is a beast.
 


Everyone is entitled to their own opinions, but that didn't stop you from calling them stupid.

All your posts (not only on this thread) indicate you're entrenched deep in Apple camp. That's fine, but quit pretending you're impartial, because you're not. Repeating it doesn't make it so, even though it may lead you to actually believe it. Everyone else here knows your affiliation. You may fool yourself, but not others.

bye
 


are you implying that because other companies do this, Samsung is exempt from criticism?
 
As I see it it's like a performance car making 600HP while on the dynamometer by detecting it's being tested, but only 500HP on the road. It's not a huge amount but I don't think that would make any car owner happy.
As I think about it, let's not give the car manufacturers any ideas :)
 
Status
Not open for further replies.

TRENDING THREADS

Latest posts

Moderators online

Share this page

COMPANY
RESOURCES
FOLLOW
Top Bottom