News AMD plans for FSR4 to be fully AI-based — designed to improve quality and maximize power efficiency

[usertests]

Something not worth "catching up" to IMO, hence my point

Upscaling is here to stay, and is continuing to improve. It's not 2019 anymore, where you can just dismiss it out of hand. 540p to 1080p would be great for battery-saving handhelds.

 

[Guardians Bane]

Ok so it’s AI based, will rdna3’s AI focused matrix extensions allow it ton on there without discrete matrix accelerators?

Considering AMD is behind on this type of feature and other software/hardware based features, it's a good thing. Nivida dominates with DLSS and Intel is pretty good too. Just because it has the AI term in it doesn't negate the improvement we will see as consumers.

These kinds of features being so far behind compared to the competition is why I won't consider a Radeon card. If I'm paying a large chunk of money on a GPU, I wasn't all the bells and whistles. Whether I end up liking the visual in real world use isn't the thing. It's having the options.

So for AMD to be focused on this and also RT in RDNA4 is freaking good! It will improve our gaming experience as it matures and if it catches and up to Nvidia and Intel then that's so much the better!

 

[Pierce2623]

Considering AMD is behind on this type of feature and other software/hardware based features, it's a good thing. Nivida dominates with DLSS and Intel is pretty good too. Just because it has the AI term in it doesn't negate the improvement we will see as consumers.

These kinds of features being so far behind compared to the competition is why I won't consider a Radeon card. If I'm paying a large chunk of money on a GPU, I wasn't all the bells and whistles. Whether I end up liking the visual in real world use isn't the thing. It's having the options.

So for AMD to be focused on this and also RT in RDNA4 is freaking good! It will improve our gaming experience as it matures and if it catches and up to Nvidia and Intel then that's so much the better!

If you honestly think there’s a significant difference in DLSS, FSR, and XeSS then you’ve fallen victim to the hype. Nvidia is ahead because they generally do everything better not because of DLSS. All three upscalers get so much information from sub-pixel detail and vectors they can all upscale a 1440p image to 4k and get a better image than native 4k coming straight out of the ROPs. Ada beats RDNA3 in every way but all modern upscaling is so good it doesn’t matter which you use. AI based models only really have benefits over FSR at the lower end of the resolution spectrum.

 

[thestryker]

If you honestly think there’s a significant difference in DLSS, FSR, and XeSS then you’ve fallen victim to the hype. Nvidia is ahead because they generally do everything better not because of DLSS. All three upscalers get so much information from sub-pixel detail and vectors they can all upscale a 1440p image to 4k and get a better image than native 4k coming straight out of the ROPs. Ada beats RDNA3 in every way but all modern upscaling is so good it doesn’t matter which you use. AI based models only really have benefits over FSR at the lower end of the resolution spectrum.

You obviously haven't actually seen the implementations in the same games. There are all sorts of random artifacting type issues and FSR is usually the worst. The most common thing that still happens with FSR is ghosting. There's a limit to how good they can get it without using a specific hardware implementation. This is why there are two versions of XeSS with the DP4a not being as good. Both FSR and XeSS have been getting a lot better overall so it stands to reason AMD is doing this for a good reason and hopefully they'll continue development of current FSR with it.

 

[Guardians Bane]

If you honestly think there’s a significant difference in DLSS, FSR, and XeSS then you’ve fallen victim to the hype. Nvidia is ahead because they generally do everything better not because of DLSS. All three upscalers get so much information from sub-pixel detail and vectors they can all upscale a 1440p image to 4k and get a better image than native 4k coming straight out of the ROPs. Ada beats RDNA3 in every way but all modern upscaling is so good it doesn’t matter which you use. AI based models only really have benefits over FSR at the lower end of the resolution spectrum.

You missed what I'm talking about. But honestly, DLSS and xess are better. But that's not my main point. If I'm going to spend let's say.... ~$800 USD on a GPU... I want the best I can get for my money. Not the one trailing behind (whether by a little or a lot).

I am in the market for a 4070ti class GPU. So when I spend that $800 or so... I want to make sure I'm getting the features that are better. Not just ok. AMD offers good GPUs when it comes to raw raster performance. Sometimes even better than Nvidia on some games.

But I also want that $800 to get me RT and frame gen and software tech and everything. AMD falls behind.

AMD is known to be redoing their RT tech from the ground up, so that's freaking awesome. But also, they're working on the AI upscaling and RT and frame gen and everything. They're not focused on raw performance this time around. They're working on catching up. And they're leveraging AI ( oh noooo, it's the hype word!!! Lol) to do most of the work on some of those technologies. Freeing up the CUs to do more efficient and better work.

I agree that the AI hype is way too much. Look at AMD's mobile APU naming... Lmao.

But there's a reason they're using it on their GPUs. That's because the AI tech really does improve overall performance for the GPU card as a whole.

So no ... I'm not falling for AI hype words. I've done my research as you can see now. I'm not going to just dismiss something because "AI" is being used.

But if it's being used as strictly a selling feature and doesn't actually add anything to the product (like Ryzen AI 370 etc...) I do laugh at it and just ignore the dumb marketing use of the term.

 

[baboma]

>If you honestly think there’s a significant difference in DLSS, FSR, and XeSS then you’ve fallen victim to the hype.

There've been a ton of DLSS/FSR/XeSS comparisons. FSR 3.1 came out the worst in all of them. Sampling below.

View: https://youtu.be/el70HE6rXV4

View: https://youtu.be/YZr6rt9yjio

View: https://youtu.be/6F3pURMsnZw

 

[JarredWaltonGPU]

>I'd like to see something where you take the last rendered frame, sample user input, and predictively generate a next frame from that. Or at the very least something like asynchronous time warp coupled to AI to get user input right before the generation of a new frame.

I don't follow. Framegen (FG) can't be predicated on user input, because user input is much slower. For 60FPS, latency is 16.7ms/frame. Per Google, avg gamer response time ~250ms, or 15 frames' time before the response is registered.

You're forgetting that up to 240Hz 4K and 500+ Hz 1080p displays exist. And also, while there's a big delay in reacting to something seem on screen, I am absolutely sure people that play a lot of games are much faster than 250ms. I can absolutely feel the difference between framegen and native rendering and the additional input lag (two frames relative to the generated fps) when a game is running at <80 fps with framegen. So that means that I, a 50 year old, can perceive a 25~37.5 ms increase in latency quite easily.

If you haven't tried framegen in a bunch of games, go give it a shot before suggesting that "user input is much slower." Because user input happens on a continuous basis, not the discrete intervals where it's sampled. So while you're moving your mouse around to aim in first person game, each rendered frame gets all the input changes since the last rendered frame. Delaying the on-screen feedback by two extra frames is noticeable.

>Basically, we need something that responds to user input somehow rather than just interpolating between two rendered frames for this to feel better and not just look better.

I agree in part. Yes, more responsiveness (ie decreased latency) is desirable, but higher framecount by itself also beneficial, because it lets you see more information in a given amount of time. It's not just for looking better.

Ex: Somebody is swinging a sword at you. In frame 1, he starts the swing; you can't dodge because you don't know where it'll land. In frame 2, the swing appears immediately above your head; you can't dodge because it's too late. If there's an intermediate frame 1.5 that shows sword in mid-swing, you can conceivably dodge it. This is independent of reaction time.

You're totally missing the point here. In your example, it would be beneficial to have an intermediate frame 1.5, sure. The problem is that with framegen, that frame 1.5 will now show up on the display after the time that the PC has already rendered frame 2, and is in fact rendering frame 3 so that it can generate frame 2.5. So now you can see, "Oh, I should have dodged four frames ago!" instead of "Oops, I should have dodge one frame ago."

What would be more beneficial is to have a higher framerate so that instead of two frames, one that's "too early" and one that's "too late" you have eight frames of data, and somewhere in the middle ground is the reaction time. But also, your contrived example isn't really something that happens in most games, because while there's a "too early" stage of rendering, the "time to react" frames usually last about a quarter to a half of a second or more so that the player has a chance to respond. (See the 250ms thing above, but then cut that number in half and add ~40ms of input to render latency.)

Nvidia has demonstrated the benefits of higher FPS (without using framegen, mind you!) on 120Hz, 240Hz, 360Hz, and 480Hz monitors. Professional gamers and even non-pro gamers did far better at hitting targets with higher frequency monitors, though of course you also need games that can render at such high framerates (which are quite rare in my experience, outside of stuff like Counter-Strike 2). What's funny is that all of the "frames win games" marketing faded away when Ada Lovelace came out and offered framegen — because Nvidia itself knows that this is walking back on responsiveness!

 

[purpleduggy]

full frame generation instead of rendering will make things once thought impossible such as PC gaming on mobile, possible. GPUs are going to transform entirely into AI accelerators. This tech if made workable on previous gen GPUs and consoles, will make hardware capable of far more.

 

[JarredWaltonGPU]

>Nvidia has demonstrated the benefits of higher FPS (without using framegen, mind you!) on 120Hz, 240Hz, 360Hz, and 480Hz monitors...What's funny is that all of the "frames win games" marketing faded away when Ada Lovelace came out and offered framegen — because Nvidia itself knows that this is walking back on responsiveness!

I assume you're referring to DLSS upscaling. Recall that when DLSS 1.0 came out in late 2018, it was obviously a work in progress. 2.0 got upscaling to "workable" in 2020. DLSS 3.0 (2022) improves upscaling further, to the point where it is now considered integral to performance, as you pointed out above. FG now is where DLSS 1.0 was in 2018.

You assume wrongly. I'm talking about high FPS delivering better results in competitive shooters. Granted, that's often super light stuff like Counter-Strike or Overwatch, but Nvidia was showing that having a high refresh rate display with a high FPS game allowed you to better respond.

The specific example was looking at a doorway, and an AI bot would run past it at random intervals. You were supposed to click to shoot the bot. At 60 Hz and 60 fps, by the time you could see the first frame showing the bot, it would already be nearly through the doorway. At 120 Hz and fps, the first frame showed up earlier and gave you more time to respond, and so on to 240 Hz and 360 Hz. This is all without DLSS or framegen.

I see FG as the logical continuation of upscaling. Whereas upscaling extends native rendering spatially (intraframe), FG extends native rendering temporally (interframe). The concept is the same. The key to FG being viable is to reduce latency to a "usable" threshold. Upscaling has reached that point, FG has not. My bet is that it will.

Fundamentally, they're related, sure, but how they get to where they end up uses very different paths. If we're running a race, upscaling is like getting everyone to run faster. Framegen is like calling a car to give you a ride to the finish line.

Interpolating a frame to smooth out the visual appearance and deliver more frames to the monitor is what framegen currently does. It will never improve responsiveness as it's not based on user input. It can't. Which is what I was getting at originally: I would like to see alternative framegen approaches that seek to accommodate user input in some fashion in order to improve not just the visually perceived fps but also the "felt fps" at the input level.

Upscaling is a pure and simple framerate booster that seeks to reduce the number of rendered pixels so that the GPU does less work. It can and does improve responsiveness because if you're rendering at 240 fps instead of 120 fps, you've just halved the rendering latency.

The latency issue is such a big and known concern with framegen that both Nvidia and AMD require games to implement latency reducing algorithms (Reflex and Anti-Lag/Anti-Lag+/Anti-Lag 2) just so that it doesn't feel quite as bad. The thing is, both of those can be done without using framegen, so really you still get worse latency.

Let's say your base system latency for a particular game is 30ms — so that's the time from input sampling to the time the CPU sends the frame data over to the GPU for final rendering. And let's say the game is running at a base 60 fps and thus has 16.7ms of latency per rendered frame. What you end up with is something like this:

Standard rendering at 60 fps:
30ms + 2 * 16.67 ms = 63.33 ms of total latency

Reflex / Anti-Lag rendering at 60 fps:
30ms + 1 * 16.67 ms = 46.67 ms of total latency

Framegen 1.5x boost rendering (ie 90 fps, with the base fps now 45):
30ms + 2 * 22.22 ms + 1 * 11.11 ms = 85.56 ms total latency

Framegen 2.0x boost rendering (ie 120 fps with 60 fps base):
30ms + 2 * 16.67 ms + 1 * 8.33 ms = 71.67 ms total latency

You'll note those are all "best-case" scenarios, and I think there are instances where framegen might even add an additional "generated framerate" of latency. It's a bit fuzzy, but I think the generated frames may have better latency than the rendered frames (or maybe it's vice versa).

Anyway, the key point is that the minimum number of frames of rendering delay will always be higher with the current interpolation approach to framegen. If you get a straight doubling of fps via framegen (it's almost always less than this, with DLSS 3 often yielding more like a 50% increase over the base fps while FSR3 seems to get closer to an 80% improvement), then at best you end up with half of an additional frame of latency (relative to the base framerate). So at 60 fps, you'd have 8.33ms more latency than the non-Reflex / non-Anti-Lag result.

Another interesting point of course is that if you're doing FSR3 framegen on a non-AMD GPU, you don't get Anti-Lag and so you end up with something like this:

AMD framegen without Anti-Lag with 1.8x boost (ie 108 fps with 54 fps base):
30ms + 3 * 18.52 ms + 1 * 9.26 ms = 94.82 ms total latency

I'm not 100% confident in all the math on these calculations, maybe only 90% confident, but at least the first four should be correct. And the last one (FSR3 without Anti-Lag) does at least match what I perceive when testing on some of the games.

So the best-case responsiveness of Reflex or Anti-Lag without framegen, you could get down to a ~50ms input lag level. That's super fast! With framegen and Reflex or Anti-Lag, you're about 20~40 percent worse input lag than standard rendering, and 70~80% higher than the base Reflex/Anti-Lag result. And framegen without Reflex or Anti-Lag, you're about double the latency of the best result.

 

[coolitic]

Upscaling is here to stay, and is continuing to improve. It's not 2019 anymore, where you can just dismiss it out of hand. 540p to 1080p would be great for battery-saving handhelds.

Spatial scaling has existed forever, and temporal scaling has existed for a long-time as well; It's nothing new, and its benefits and limitations have been known for a long time. It's just been overhyped w/ branding recently due to a band-aid being needed for increasingly poorly-performing games. The only thing that's actually new is "optical flow" upscaling; and technically the usage of AI to "enhance" other forms of scaling as well, but the benefits of the latter are minor and comes w/ a bit of its own issues.

Spatial upscaling has no real negative side-effects other than the slight blur of non-fractional scaling (partially offsetable w/ sharpening, which has its own issues), and temporal scaling of *any* kind has negative side-effects that are endemic to how it works.

 

[Pierce2623]

You obviously haven't actually seen the implementations in the same games. There are all sorts of random artifacting type issues and FSR is usually the worst. The most common thing that still happens with FSR is ghosting. There's a limit to how good they can get it without using a specific hardware implementation. This is why there are two versions of XeSS with the DP4a not being as good. Both FSR and XeSS have been getting a lot better overall so it stands to reason AMD is doing this for a good reason and hopefully they'll continue development of current FSR with it.

I use every upscaling option available on an Nvidia GPU in every game. I often find I prefer the extra performance of FSR over DLSS in CPU bound titles.

 

[JarredWaltonGPU]

Spatial scaling has existed forever, and temporal scaling has existed for a long-time as well; It's nothing new, and its benefits and limitations have been known for a long time. It's just been overhyped w/ branding recently due to a band-aid being needed for increasingly poorly-performing games. The only thing that's actually new is "optical flow" upscaling; and technically the usage of AI to "enhance" other forms of scaling as well, but the benefits of the latter are minor and comes w/ a bit of its own issues.

Spatial upscaling has no real negative side-effects other than the slight blur of non-fractional scaling (partially offsetable w/ sharpening, which has its own issues), and temporal scaling of *any* kind has negative side-effects that are endemic to how it works.

Eh... spatial upscaling has a LOT of negatives. Specifically, it just doesn't look anywhere near as good as native rendering. If you upscaling from maybe 90% it can come close, and if you start with a poor baseline (like TAA 100% scaling with overly aggressive blur) it might be "better." But FSR1 and various other spatial upscaling algorithms — including DLSS 1.x — just never looked that great.

And "optical flow upscaling" isn't a thing. The Optical Flow Accelerator in the RTX 40-series is exclusively used for frame generation, not upscaling. (It can also do some stuff for video, but that's a different topic. The OFA has been around since RTX 20-series, but it has become substantially more potent over time, and Nvidia hasn't really divulged much about where the OFA helped out before other than a nebulous "in video" comment.)

 

[Pierce2623]

Eh... spatial upscaling has a LOT of negatives. Specifically, it just doesn't look anywhere near as good as native rendering. If you upscaling from maybe 90% it can come close, and if you start with a poor baseline (like TAA 100% scaling with overly aggressive blur) it might be "better." But FSR1 and various other spatial upscaling algorithms — including DLSS 1.x — just never looked that great.

And "optical flow upscaling" isn't a thing. The Optical Flow Accelerator in the RTX 40-series is exclusively used for frame generation, not upscaling. (It can also do some stuff for video, but that's a different topic. The OFA has been around since RTX 20-series, but it has become substantially more potent over time, and Nvidia hasn't really divulged much about where the OFA helped out before other than a nebulous "in video" comment.)

Isn’t “optical flow” basically just fancy talk for the vectors used in upscaling/decoding/encoding with a little extra info thrown in ?

 

[JarredWaltonGPU]

Isn’t “optical flow” basically just fancy talk for the vectors used in upscaling/decoding/encoding with a little extra info thrown in ?

Yes, temporal upscaling has an “optical flow” but the 40-series specifically has an “Optical Flow Accelerator” that handles framegen. I’m not totally certain, but I think it just takes two frames and computes an optical flow from that. Or it’s possible that it gets the depth buffer and motion vectors from the game as well… but given modders have stuffed DLSS3 framegen into games that don’t already have DLSS support, I think it was designed as a black box that handles everything.