@JarredWaltonGPU
Very interesting review, this looks like a very capable piece of hardware.
Can this thing cast (wirelessly ofc) its picture/sound to a TV?
If so, have you tried it out? Is it viable?
You'd have to ask
@AndrewFreedman about that, as he's the one with the Ally hardware.
Anyway, to the article. From what I'm reading and comparing your results to others, I have to question methodology or due diligence in making sure everything was up to snuff in terms of updates, background processes etc. It's that, or it seems like you guys had a defective unit. Or everyone else is lying or wrong with their benchmarks.
I get that that is a part of testing new hardware, and sometimes you have 1 unit and stuff isn't right and you can only go with what your seeing. However, were you allowed to reach out to other outlets and validate your results?
It's possible that the problems all stem from having a defective unit. However, that in itself would be concerning. Sending out review units that are defective suggests a lack of QA. This is why the review starts with a note about the problems Andrew experienced, and until he has a replacement unit, we don't know for certain how it will perform.
Still, I don't see how a hardware site that regularly covers silicon can look at the spec sheet of the Z1 extreme and not see any issue in those results comparative to the decks Van Gogh. You're basically getting worse than 6800u performance.
Power constrained devices can behave in unexpected ways. Differences between tech specs and real-world performance are the whole reason to run benchmarks! Most places that actually bothered to run benchmarks, rather than just showing a game running, provided hard data (as in a summary average FPS result) for the Ally in both performance and turbo modes.
Just as an example, ETA PRIME showed CP2077 running on battery 1080p low settings set to 15w getting just shy of 60 FPS average. Then 1080p low settings in turbo mode averaging 74 FPS.
RDR2 1080p low settings (favor performance), in performance mode he got 44 FPS average and in turbo mode he got 64 FPS average.
Forza Horizon 5 at 1080p medium settings at 15w he showed 81 FPS average and in turbo mode he showed 101 FPS average.
He was actively playing these games on the ally in his video, with the device running on battery.
That last line is the most important detail. He wasn't running a repeatable benchmark, and he didn't show the exact same test and settings with the Steam Deck. We really have no idea what he was doing, but with a video review title of "This Changes Everything" I can't help but think he's chasing views rather than being fully objective.
He was just playing games with an FPS overlay in the corner. There's literally no comparison given with any other hardware that I saw while skimming the video. I'm not questioning the FPS counter here, but rather pointing out that different areas of a game can behave quite differently.
For example, Forza Horizon 5's built-in benchmark is a full race, lasting about 90 seconds, where there are always multiple other cars on the screen (probably at least five in all cases, and as many as 10-11 in some cases). Driving around in the open world portion of the game with one or two other cars occasionally visible is not the same. Period.
The same applies to Red Dead Redemption 2 and Cyberpunk 2077. Plus, besides him saying the games were running at 1080p Low, I have no idea if that's actually correct or what those settings mean. If the settings we (Andrew) used aren't the same as what ETA Prime used, you can again get wildly different results. If you're doing a video, at least show the settings screen and then drop into the game!
Benchmarks need to be repeated in the same fashion, with the same settings. We couldn't use the same resolution for both the Steam Deck and Ally, but the test sequences were the same. An FPS counter in a corner isn't actually a benchmark. You need to log frametimes, then convert to average FPS values. Or at least use a repeatable benchmark that does that for you (which is what Andrew did). Because sometimes things get missed if you're just watching a video with an FPS counter, doubly so if you're just running around in a game without following the same path each time.
There are definitely other reviews that show middling performance, especially in the default Performance mode. Trusted Reviews for example has
these five charts. At 720p, in performance mode, those show the Ally was 15% faster in Horizon Zero Dawn, 18% faster in Dirt Rally, 5% faster in Cyberpunk 2077, 18% faster in Returnal, and 24%
slower in F1 2022.
The Verge review: Ally was 23% faster in Cyberpunk, 33% faster in Deus Ex: Mankind Divided, 20% faster in Elden Ring, and 11% faster in Shadow of the Tomb Raider. Again, that's comparing Performance mode, because Turbo mode cuts battery life quite a bit.
If we toss out F1 2022 in the Trusted Reviews as an outlier, comparing Ally in Performance mode to the Steam Deck, we get an average FPS advantage of 21% (The Verge), 15% (Trusted Reviews), and 17% (Tom's Hardware). I should also note that technically we at least are comparing 1280x720 on the Ally to 1280x800 on the Steam Deck, so the Steam Deck is rendering 11% more pixels, which means you could probably subtract 5–10 percent from our results if you want true equivalent performance comparisons at the same resolution . (That might apply to the other two reviews I referenced as well, but I don't know for certain what settings they used.)
Clearly, our numbers aren't all that different from what I found in a quick Google search for "Asus Ally performance" — Trusted Reviews was the top result, then The Verge, then PC Gamer (which did asinine ultra ray tracing performance comparisons with laptops for whatever reason, so there weren't any useful data points in there).
This is what being objective is like. You give reference points to other competing hardware, and you show performance in a like-for-like comparison. I don't really care if you love the Ally (even though you haven't used it, AFAIK), but that's obviously not an objective statement or a review.
Facebook
Twitter
Instagram