News Qualcomm faces benchmark cheating allegations — Snapdragon X Elite/Plus benchmarks claimed to be fraudulent

[Admin]

SemiAccurate reports that it has independently determined that Qualcomm's freshly announced laptop chips are being marketed erroneously to consumers and OEMs.

Qualcomm faces benchmark cheating allegations — Snapdragon X Elite/Plus benchmarks claimed to be fraudulent : Read more

 

[ThomasKinsley]

I read the article by SemiAccurate. It can be distilled (by me, not AI) into three points:

1. Qualcomm has not been forthcoming with detailed specs of their chip.
2. OEMs reported poor performance with samples - sometimes 50% slower; other times much more.
3. An insider at Qualcomm claims they cooked the benchmarking, yet at the same time by manipulating the benchmarks they made the chip look worse in other areas than it really is.

On the first two points SemiAccurate states that Qualcomm worked with them months ago. They provided more information on the chip and blamed Windows on ARM for being unoptimized.

I'm trying to see every side here. It could be Qualcomm is maliciously lying about their benchmarks, but if it's really as slow as a Celeron (as one OEM stated), then that will be discovered very quickly by tech journalists.

What if Qualcomm is struggling to get the chip to play nicely with Windows 11 on Arm? That would make sense why Qualcomm is delaying the benchmarks and test samples to journalists. Nobody at Qualcomm wants to hear, "The Snapdragon X Elite runs like trash! What a loser!" when they believe it's MS's fault. Even if MS fixes their OS after launch, everyone will still think of the SD X Elite as junk and the initial image can't be fixed. That might explain it all.

 

[Bruhme2985]

I checked the article tonshardware linked and man did it sound incredibly petty and childish. The website for one doesn't look even remotely professional and the article is just a dude ranting that he didn't get an inside peek at it.

A lot of things including the fact that the dude messed things up in the article (for instance snapdragon doesn't have an x Pro chipset), shows that it is more of a rant then anything special, the dude wasn't even bothered to reread his own rant.

Honestly his article imo has very little weight to it, even if it was true his article is too unprofessional and salty. It makes sense why snapdragon doesn't want to talk to you, you sound too bitter.

 

[bcweir]

Wait, you call the author's remarks petty and childish, yet refer to the other person as 'dude,' undercutting your own argument to treat people with respect. Unless you're either a Valley Girl or a professional surfer, try following your own rules before calling out someone else for the same sins you're committing

 

[bit_user]

I checked the article tonshardware linked and man did it sound incredibly petty and childish. The website for one doesn't look even remotely professional and the article is just a dude ranting that he didn't get an inside peek at it.

I can confirm that SemiAccurate (mostly the work of Charlie Demerjian) has fallen far from whatever grace it had. The articles try way too hard to strike a clever and informal tone, while self-congratulating and calling out every slight the author believes he received, at every possible opportunity. It turns them into rants that I find virtually unreadable. Add to that the jealous paywall-guarding of the few actual nuggets of information the author managed to collect and the entire site is basically just a frustrating waste of time.

I really don't know why anyone in the industry still talks to Charlie, at this point. It's probably just a shrinking number of disgruntled tech workers trying to settle a score, which makes his information very subject to exaggeration and mischaracterization.

If you want to see what SemiAccurate could have been, head on over to SemiAnalysis:

SemiAnalysis

Bridging the gap between the world's most important industry, semiconductors, and business.

www.semianalysis.com

 

[bit_user]

Oh, and as for the allegations, I'm probably not as bothered by them as I should be. I try to disregard both leaks and curated benchmark scores, instead just waiting until review samples get into the hands of independent tech journalists.

That said, even the best review I saw of the Thinkpad X13S, which I think is probably the leading implementation of the Snapdragon 8cx Gen 3 we've seen to date, is still quite lacking in benchmarks of native apps, even where they exist (i.e. web, productivity, etc.). So, I do understand Qualcomm's concerns about how it will be treated by the tech press and influencers looking to score points with their audience at its expense.

 

[Alvar "Miles" Udell]

You mean a first party manufacturer is presenting numbers that were probably obtained in a very specific way on order to maximize the performance numbers for publicity? That NEVER happens with AMD, Intel, nVidia, or any other company...

 

[JamesJones44]

Assuming everything SemiAccurate said ends up being true, the minute independent benchmarks come out Qualcomm would be in big trouble. Maybe they are crazy enough to try, but that seems like the fastest way to have your product killed and lawsuits thrown at your company.

Guess we'll find out in the July timeframe.

 

[Bruhme2985]

Wait, you call the author's remarks petty and childish, yet refer to the other person as 'dude,' undercutting your own argument to treat people with respect. Unless you're either a Valley Girl or a professional surfer, try following your own rules before calling out someone else for the same sins you're committing

I don't know why you are getting triggered over the word dude. It's informal English, and the guy is a dude a male. I am a dude, a male human. How is dude derogatory, it really isn't.

 

[Bruhme2985]

I can confirm that SemiAccurate (mostly the work of Charlie Demerjian) has fallen far from whatever grace it had. The articles try way too hard to strike a clever and informal tone, while self-congratulating and calling out every slight the author believes he received, at every possible opportunity. It turns them into rants that I find virtually unreadable. Add to that the jealous paywall-guarding of the few actual nuggets of information the author managed to collect and the entire site is basically just a frustrating waste of time.

I really don't know why anyone in the industry still talks to Charlie, at this point. It's probably just a shrinking number of disgruntled tech workers trying to settle a score, which makes his information very subject to exaggeration and mischaracterization.

If you want to see what SemiAccurate could have been, head on over to SemiAnalysis:

​

SemiAnalysis

Bridging the gap between the world's most important industry, semiconductors, and business.

www.semianalysis.com

​

Alright. Man, I wonder what happened to him that made him change I guess. Maybe stress or something. I read the article and it was very weird, he acted as if qualcomn owed him to get a sneek peak of their chip.

 

[CmdrShepard]

In the past Charlie was right on Intel SNAFUs so I will give them a benefit of a doubt (I don't trust Qualcomm as far as I can throw them).

 

[Snapperkins]

I don't know why you are getting triggered over the word dude. It's informal English, and the guy is a dude a male. I am a dude, a male human. How is dude derogatory, it really isn't.

I'm a dude. He's a dude. She's a dude. 'Cause we're all dudes... Hey!

 

[watzupken]

I guess we are not that far from knowing the actual performance of these new Qualcomm chips. The launch of laptops with these processors I believe should be a couple of months away if I am not mistaken since they mentioned mid of the year. Ultimately you can cook what you want up, but when the actual product goes out to the market, it will become obvious whether Qualcomm is being truthful about the performance number.

I do think that Qualcomm's effort is mostly going to be hampered by Windows optimization. Apple's M1 success was due to a combination of good hardware and software optimization.

 

[sonichedgehog360]

In the past Charlie was right on Intel SNAFUs so I will give them a benefit of a doubt (I don't trust Qualcomm as far as I can throw them).

Charlie has fallen from grace significantly from where he used to be during his heyday in the mid 2010s when he had quality contacts in the industry. His site used to even have a web forum but he was too lazy and incompetent to ever get it back online, eventually nixing it and flushing away all of his subscribers' content down the toilet. That incident alone should tell you how technically savvy the man is when he cannot even get a web forum online for his subscribers, many of whom dish out $1500 a year for his "exclusive" leaks.

AnandTech's own Andrei Frumusanu now does performance modeling for NUVIA at Qualcomm. While at AnandTech, he was widely known as the go-to guy in the tech journalism circle for performance analysis of CPU microarchitectures. So I would take smarty Andrei's NUVIA work, which likely we are seeing reflected in some degree or another in Qualcomm's official slide decks, over salty Charlie's keyboard warrioring rants any day of the week. Wake me up when Charlie talks figures and percentages in that cited article instead of obtuse, old-man-yells-at-cloud non-specifics.

As @bit_user mentioned, SemiAnalysis is an excellent source for thorough nuanced and civil discussion on industry dynamics and trends. To piggy back off of that, I would also recommend Chips and Cheese.

 

[cyrusfox]

Charlie is fun to read, maybe more valid sources than MLID, maybe worse🤷‍♂️, but he has been grinding an axe against Qualcomm for a couple years now. It was sad when his Forum went away, as well as his website going down to just him as the sole writer.

Online Tech journalism has been on a continual slow death decline for the last 10 years. I remember frequenting Anandtech, Fudzilla, Toms, Semiaccurate, and a host of other now dead platforms. Anandtech and Toms have held on the longest and still are relevant articles and forums. Reviews are missing the more detailed, indepth analysis of prior articles. Anandtech still makes the best tables and are a great historic reference (especially for fail products!). Toms has the best interactive platform(edit comments on articles). Fudzilla is just recyled content (Charlie calls it the echo chamber).

What Charlie produces is always unique, I don't pay for the paywall, so I have no idea how useful the full information is, but he does a decent job teasing for that subscription and I would not doubt he still has a pulse on the industry and still extensive contacts. With how phenomenal the initial Qualcomm numbers looked, this provides a counterpoint as well as gives x86 companies some room to breath if true. Before it looked like a slaughter comparing Qualcomm to Intel/AMD mobile offerings.

 

[thestryker]

AnandTech's own Andrei Frumusanu now does performance modeling for NUVIA at Qualcomm. While at AnandTech, he was widely known as the go-to guy in the tech journalism circle for performance analysis of CPU microarchitectures. So I would take smarty Andrei's NUVIA work, which likely we are seeing reflected in some degree or another in Qualcomm's official slide decks, over salty Charlie's keyboard warrioring rants any day of week.

On this note Ian Cutress strongly implied the first benchmarking Qualcomm released on Snapdragon X was Andrei's work.

 

[Spamurai71]

Considering that the SD 8 Gen 3 gets more or less double the performance of the Intel N100 in benchmarks, the claim that the SD X has Celeron level performance is highly dubious.

 

[bit_user]

Online Tech journalism has been on a continual slow death decline for the last 10 years.

igorslab.de has been doing pretty well, since its founder (Igor Wallossek; forum user: FormatC) parted ways with this site. I find the articles on there & computerbase.de are quite readable, via Google Translate.

igor´sLAB | PC & Components | Reviews & News

www.igorslab.de

Also, I just noticed TechPowerUp is even hiring:

https://www.techpowerup.com/320826/...aptops-gaming-handhelds-and-prebuilt-desktops
​

Perhaps it's not a good sign that article is dated more than 3 weeks ago and still featured on their front page.

What Charlie produces is always unique, I don't pay for the paywall, so I have no idea how useful the full information is, but he does a decent job teasing for that subscription

I guess if you like being teased...

What I never understood is why he doesn't make his articles free, after a couple years. He likes to pat himself on the back and tell us he was right, but unless he actually puts any of that stuff outside the paywall, we just have to take his word for it.

 

[abufrejoval]

I can confirm that SemiAccurate (mostly the work of Charlie Demerjian) has fallen far from whatever grace it had. The articles try way too hard to strike a clever and informal tone, while self-congratulating and calling out every slight the author believes he received, at every possible opportunity. It turns them into rants that I find virtually unreadable. Add to that the jealous paywall-guarding of the few actual nuggets of information the author managed to collect and the entire site is basically just a frustrating waste of time.

I really don't know why anyone in the industry still talks to Charlie, at this point. It's probably just a shrinking number of disgruntled tech workers trying to settle a score, which makes his information very subject to exaggeration and mischaracterization.

If you want to see what SemiAccurate could have been, head on over to SemiAnalysis:

​

SemiAnalysis

Bridging the gap between the world's most important industry, semiconductors, and business.

www.semianalysis.com

​

Unfortunately I have to agree 100%, Charlie has become really painful to read and also to watch.

Over the last few HotChips conferences his questions stood out by essentially being loaded with accusations before even asking a single meaningful detail and sometimes I had even the impression that he hadn't listened to the the talk.

I also agree with SemiAnalysis, although they can lean quite far out of the window, which makes it fun to re-read their older predictions.

I wouldn't rely on them for buying or selling stock, but in terms of what-if ruminations, they are hard to beat.

They are also very educational, not too far off for a software guy like me, who hasn't lived inside silicon for decades.

And I consider the price for their paywall quite ok for what you get; an impression I never remotely had with SemiAccurate.

 

[Pete Mitchell]

Assuming everything SemiAccurate said ends up being true, the minute independent benchmarks come out Qualcomm would be in big trouble. Maybe they are crazy enough to try, but that seems like the fastest way to have your product killed and lawsuits thrown at your company.

Guess we'll find out in the July timeframe.

It's not crazy. Intel, Nvidia, and AMD have all done it and gotten caught.

If the allegations are true I would expect that the marketing department was behind it, expecting that engineering could hit near those benchmarks on production systems. If they hadn't published those crazy numbers that were comparable to Apple, AMD, and Intel, everyone would have been like "meh, another mediocre Qualcom chip", and the tech press wouldn't have bothered reporting on it.

 

[abufrejoval]

igorslab.de has been doing pretty well, since its founder (Igor Wallossek; forum user: FormatC) parted ways with this site. I find the articles on there & computerbase.de are quite readable, via Google Translate.

​

Igor's site used to be quite a bit better when it was only him publishing. For some time now he's been adding content by other authors to gain mass appeal and those articles are mostly re-hashes that make it harder to find his gems.

Me no need Google translate for German, but I believe he's using professional tools to publish 100% of his stuff in English now: some of his self-produced earlier English content was quite painful to read, clearly reflecting his upbringing in East-Germany, where English was very grudgingly taught by teachers who had to pass the most intensive political screening.

Another one on my daily reading list is Borja, mostly because he can be quite fast (but also somewhat sloppy) on things: I then tended to follow things up on Anandtech days or weeks later for validation, and mostly came here because that site has become a bit of a ghost town.

 

[abufrejoval]

By far the biggest issue I see with these ultrabook snapdragons is that they seem to be Windows exclusively, no signs at all of any Linux support.

And I distinctly remember that the earliest ARM Windows variants forced use of the M$ store.

Without Linux I wouldn't touch them with a long pole, nobody needs another fruity cult.

 

[bit_user]

By far the biggest issue I see with these ultrabook snapdragons is that they seem to be Windows exclusively, no signs at all of any Linux support.

Not sure about the upcoming generation, but it seems Qualcomm has been devoting some resources to Linux support.

https://www.phoronix.com/news/Qualcomm-Mainline-Linux-2024
​

 

[usertests]

I checked the article tonshardware linked and man did it sound incredibly petty and childish. The website for one doesn't look even remotely professional and the article is just a dude ranting that he didn't get an inside peek at it.

SemiAccurate / Charlie Demerjian definitely has had a huge hate boner for Qualcomm for years, with this being one of the quintessential articles:

Qualcomm 8cx Gen 3: Too dangerous to deploy

Problem is, I don't think their criticisms miss the mark. Qualcomm is a bad company pushing the latest in a string of failed ARM PC attempts with suspicious claims, eagerly bolstered by an "x86 is dying" crowd. That being said, I have not read this new article yet, will get around to it a bit later.

 

[JarredWaltonGPU]

I can confirm that SemiAccurate (mostly the work of Charlie Demerjian) has fallen far from whatever grace it had. The articles try way too hard to strike a clever and informal tone, while self-congratulating and calling out every slight the author believes he received, at every possible opportunity. It turns them into rants that I find virtually unreadable. Add to that the jealous paywall-guarding of the few actual nuggets of information the author managed to collect and the entire site is basically just a frustrating waste of time.

I really don't know why anyone in the industry still talks to Charlie, at this point. It's probably just a shrinking number of disgruntled tech workers trying to settle a score, which makes his information very subject to exaggeration and mischaracterization.

If you want to see what SemiAccurate could have been, head on over to SemiAnalysis:

​

SemiAnalysis

Bridging the gap between the world's most important industry, semiconductors, and business.

www.semianalysis.com

​

Pretty much this. I've always felt the most honest thing about the site was its name: semi-accurate. That "semi" bit often falls far below 50% in my experience. Charlie hasn't really changed all that much since his early days at The Inquirer, and often seems to have a major axe to grind. I can't imagine paying for access to the deeper content he promises, but then some people love the drama and fighting. 🤷‍♂️