Question What GPU Benchmarks Should I Add/Drop for Future Reviews/Hierarchy?

JarredWaltonGPU

Senior GPU Editor
Editor
Feb 21, 2020
697
570
1,270
1
Here's your chance to influence our future graphics card testing. Right now, our list of nine games that we use for GPU benchmarks consists of the following (parenthetical comments are API we use, and whether a game has AMD or Nvidia branding):

Borderlands 3 (DX12 AMD)
The Division 2 (DX12 AMD)
Far Cry 5 (DX11 AMD)
Final Fantasy XIV (DX11 Nvidia)
Forza Horizon 4 (DX12)
Metro Exodus (DX12 Nvidia)
Red Dead Redemption 2 (Vulkan)
Shadow of the Tomb Raider (DX12 Nvidia)
Strange Brigade (Vulkan AMD)

A few quick notes:
  1. I really want to ditch Far Cry 5, FFXIV, and Strange Brigade -- I feel all are older and less meaningful representations of modern games. FFXIV in particular has got to go IMO.
  2. I haven't been testing with any ray tracing or DLSS modes in the two games that support those features (Metro and SotTR), but when AMD's Big Navi arrives I'll very likely be looking to include at least two ray tracing games.
  3. The final list of games should include a mix of AMD and Nvidia promoted titles, different APIs if possible (DX12 + Vulkan for sure, and a major DX11 game would be nice as well). It should also include a mix of genres if possible -- so Forza is currently my "car racing" benchmark, and I still feel it's better than the F1 20xx games ... but if you have strong feelings and want to argue the case for a different racing game, I'm open.
  4. Ideally, I want games that are more demanding (on the GPU), better looking, and more popular -- so if a game is more demanding but no one plays it, I'd prefer not to use it.
My short list of games I may include when they become available:

Assassin's Creed Valhalla (AC games are usually very demanding and decently popular, but potentially buggy)
Cyberpunk 2077 (duh)
Dirt 5 (could be a reasonable swap out for FH4)
Doom Eternal (it's out, I've tested it already, but it takes more effort to benchmark)
Dying Light 2 (maybe -- the last one was a pain to benchmark)
Microsoft Flight Simulator (if there's a reliable way to test that's not a complete PITA)
Rainbow Six Quarantine (another big maybe)
Star Wars Squadrons (maybe -- Jedi Fallen Order was merely okay, didn't warrant inclusion IMO)
Vampire: The Masquerade -- Bloodlines 2 (if it doesn't suck)
Watchdogs: Legion (possibly)

Lots of other games I probably won't include, unless they end up being 'better' for testing GPUs:
Death Stranding (already out, not sure how 'relevant' it is -- weird game, pretty repetitive IMO, but maybe?)
Diablo 4 (probably not coming for a while, may not be very demanding graphically)
Gods and Monsters (maybe?)
Horizon Zero Dawn (belated PS4 port)
Marvel's Avengers (likely to be too 'light' IMO)
Project Cars 3 (I didn't do much with the previous 2, but maybe third time's the charm)
Serious Sam 4 (will it be any good, and will it push GPUs at all?)
Wasteland 3 (Unity engine, not super demanding most likely)
Werewolf: The Apocalypse - Earthblood (probably not going to be demanding or popular?

I'd like to keep the final list to no more than 10 games total. Games with built-in benchmarks are also lovely, because that can remove a lot of the difficulty of repeated testing and variability -- but I get why some prefer 'real' benchmarking. Anyway, that's it. Sound off and I'll be looking at responses to see if I need to seriously rethink anything I've listed above.
 
Aug 25, 2020
6
2
15
0
Would recommend you to (re)include Warframe. Their latest open world (Cambion Drift) is quite demanding, much more than any other part of the game, and it's also quite beautiful.
 
Reactions: Liisel007

hotaru.hino

Prominent
Sep 1, 2020
829
250
790
19
For Microsoft Flight Simulator, supposedly you can have the AI fly a route. So at the very least you can have some repeatability as long as the other conditions are the same. The only thing you may have to turn off is cloud rendering since network hiccups could screw things up.
 
Sep 10, 2020
4
1
15
0
This might sound odd, but with the new 3080 FE thermal design, I'm very interested to see how case, CPU, and RAM temps will be impacted by the pass through FE fan that will blow air right up to the CPU.

Creating a standard platform for all air cooling and testing with open and closed case with both Intel and AMD and various CPU coolers would be a tremendous asset in deciding on how to proceed with a build using the new card. That might be a big ask, but that would be incredibly useful.

It would be even better if different ambient temps could be applied to the test PC to see thermal profiles.
 

1foxracing

Distinguished
Feb 14, 2011
119
7
18,695
4
I second the Blender software test request but what I would really like to see id 4k Video rendering test with Vegas Pro17. Don't care about Adobe and all their buggy software period.
To me it's absolutely ludicrous to spend $500 let alone $1000 or $1500 on a graphics card to "play games" I use my computers to make money with photography and video work.
 

chfosmith

Distinguished
Jun 25, 2009
5
0
18,510
0
Here is another vote for adding Microsoft Flight Simulator 2020 at 4K.
When the settings are at Ultra, the scenery is complex, and the clouds are fully defined, it will bring any GPU to its knees. (Less than 60fps)
This means much more powerful GPUs in the future can be properly evaluated.
It must be mentioned that Microsoft has already sold millions of copies of MFS 2020 world wide, so there will be a lot of interest in GPU ratings for this game.
 
Having MMOs, like you had WoW in the past, would be nice.

Most MMOs still use DX9c and while they don't require expensive setups, they are still the popular game people plays. Having benchmarks on crowded areas would be a good thing and/or maps with a lot of things going on (like battlegrounds or big PvP areas).

Cheers!
 

Umeed

Distinguished
Apr 16, 2013
164
1
18,715
5
Here's your chance to influence our future graphics card testing. Right now, our list of nine games that we use for GPU benchmarks consists of the following (parenthetical comments are API we use, and whether a game has AMD or Nvidia branding):

Borderlands 3 (DX12 AMD)
The Division 2 (DX12 AMD)
Far Cry 5 (DX11 AMD)
Final Fantasy XIV (DX11 Nvidia)
Forza Horizon 4 (DX12)
Metro Exodus (DX12 Nvidia)
Red Dead Redemption 2 (Vulkan)
Shadow of the Tomb Raider (DX12 Nvidia)
Strange Brigade (Vulkan AMD)

A few quick notes:
  1. I really want to ditch Far Cry 5, FFXIV, and Strange Brigade -- I feel all are older and less meaningful representations of modern games. FFXIV in particular has got to go IMO.
  2. I
I'd say get rid of all of that AND shadow of the tomb raider. There's also a free bench mark tool that you can get from Steam it's called "Bright Memory Infinite Ray Tracing Benchmark". It's a fairly quick one that seems to focus on RTX and DLSS built in Unreal Engine, might be worth checking out? I used my 3080 ftw3 ultra on it and at 4k res got an avg 36fps with rtx = very high, and DLSS = quality lol

As for games themselves, I do think Anthem is worth a shot at least from a visual gpu driving perspective. Anthem 2.0 (possible no man's sky style redeemer?) will be a good option.
 

Akira1364

Prominent
Aug 31, 2019
5
0
510
0
One general comment about the most recent version of the GPU Hierarchy... it seems like the positions of the 1660 Ti and 1660 Super were (accidentally?) swapped.

It makes no sense for the Super to be ahead... that would contradict this site's very own review of the 1660 Super, as well as basically all other available comparative data between the two cards from other reviewers. They're not hugely different, but the Ti definitely has the overall edge due to having more CUDA cores and more texture mapping units.
 

JarredWaltonGPU

Senior GPU Editor
Editor
Feb 21, 2020
697
570
1,270
1
One general comment about the most recent version of the GPU Hierarchy... it seems like the positions of the 1660 Ti and 1660 Super were (accidentally?) swapped.

It makes no sense for the Super to be ahead... that would contradict this site's very own review of the 1660 Super, as well as basically all other available comparative data between the two cards from other reviewers. They're not hugely different, but the Ti definitely has the overall edge due to having more CUDA cores and more texture mapping units.
The two cards were always very close. 1660 Ti has more cores, yes, but 1660 Super has faster GDDR6 memory and slightly higher GPU clocks. Some games favor the 1660 Ti by a few points, others favor the 1660 Super. I’m not actually sure why they swapped places but they’re effectively tied — like a 0.1 FPS difference once all the numbers are added together.

Actually, I think my weighting for the hierarchy used average of all FPS before, and in the last month I swapped to geometric mean in the spreadsheet. And that was enough for a 0.3 FPS change (because the 1660 Ti used to be slightly ahead). Of course, the GDDR6 memory on the Ti can probably hit similar overclocks and that would push it ahead, but I don’t think Nvidia is even making those anymore. It decided 1660 Super was preferable, possibly for yield and cost reasons.
 

bohemond1099

Prominent
Oct 6, 2019
28
3
545
2
Here's your chance to influence our future graphics card testing. Right now, our list of nine games that we use for GPU benchmarks consists of the following (parenthetical comments are API we use, and whether a game has AMD or Nvidia branding):

Borderlands 3 (DX12 AMD)
The Division 2 (DX12 AMD)
Far Cry 5 (DX11 AMD)
Final Fantasy XIV (DX11 Nvidia)
Forza Horizon 4 (DX12)
Metro Exodus (DX12 Nvidia)
Red Dead Redemption 2 (Vulkan)
Shadow of the Tomb Raider (DX12 Nvidia)
Strange Brigade (Vulkan AMD)

A few quick notes:
  1. I really want to ditch Far Cry 5, FFXIV, and Strange Brigade -- I feel all are older and less meaningful representations of modern games. FFXIV in particular has got to go IMO.
  2. I haven't been testing with any ray tracing or DLSS modes in the two games that support those features (Metro and SotTR), but when AMD's Big Navi arrives I'll very likely be looking to include at least two ray tracing games.
  3. The final list of games should include a mix of AMD and Nvidia promoted titles, different APIs if possible (DX12 + Vulkan for sure, and a major DX11 game would be nice as well). It should also include a mix of genres if possible -- so Forza is currently my "car racing" benchmark, and I still feel it's better than the F1 20xx games ... but if you have strong feelings and want to argue the case for a different racing game, I'm open.
  4. Ideally, I want games that are more demanding (on the GPU), better looking, and more popular -- so if a game is more demanding but no one plays it, I'd prefer not to use it.
My short list of games I may include when they become available:

Assassin's Creed Valhalla (AC games are usually very demanding and decently popular, but potentially buggy)
Cyberpunk 2077 (duh)
Dirt 5 (could be a reasonable swap out for FH4)
Doom Eternal (it's out, I've tested it already, but it takes more effort to benchmark)
Dying Light 2 (maybe -- the last one was a pain to benchmark)
Microsoft Flight Simulator (if there's a reliable way to test that's not a complete PITA)
Rainbow Six Quarantine (another big maybe)
Star Wars Squadrons (maybe -- Jedi Fallen Order was merely okay, didn't warrant inclusion IMO)
Vampire: The Masquerade -- Bloodlines 2 (if it doesn't suck)
Watchdogs: Legion (possibly)

Lots of other games I probably won't include, unless they end up being 'better' for testing GPUs:
Death Stranding (already out, not sure how 'relevant' it is -- weird game, pretty repetitive IMO, but maybe?)
Diablo 4 (probably not coming for a while, may not be very demanding graphically)
Gods and Monsters (maybe?)
Horizon Zero Dawn (belated PS4 port)
Marvel's Avengers (likely to be too 'light' IMO)
Project Cars 3 (I didn't do much with the previous 2, but maybe third time's the charm)
Serious Sam 4 (will it be any good, and will it push GPUs at all?)
Wasteland 3 (Unity engine, not super demanding most likely)
Werewolf: The Apocalypse - Earthblood (probably not going to be demanding or popular?

I'd like to keep the final list to no more than 10 games total. Games with built-in benchmarks are also lovely, because that can remove a lot of the difficulty of repeated testing and variability -- but I get why some prefer 'real' benchmarking. Anyway, that's it. Sound off and I'll be looking at responses to see if I need to seriously rethink anything I've listed above.
I'd say what would be most useful in the current pit of ridiculous prices, would be a look at what can be done with older GPUs; i.e. 770 4GB/780/290/290X780Ti/970/980/1050Ti @1080P/1440P low-high depepending on the game; and inclusion of the most played games. Like CSGO and PUBG.
 
Reactions: AntonyLovric

aalkjsdflkj

Commendable
Jun 30, 2018
15
2
1,515
0
I'd love to see some comparison of VR performance. I've heard that NVIDIA is better with VR, but I don't know if that's true or not, and if so if AMD has addressed the difference with the 6xxx series. I don't have any particular game or test to suggest. Even if there's a generic test tool that can be used to generate scores for different cards that would be very helpful.
 

AntonyLovric

Honorable
Aug 4, 2014
234
13
10,715
23
I like the CSGO and PUBG comment, but I realize the # of players is dropping and you might not get that much interest / views if the titles are not as popular. I do think you should include an RTX enhanced game that's GPU intensive.
 

ASK THE COMMUNITY

TRENDING THREADS