Question Difference between frame generation and DLSS

[ridwan123]

I just wondering whats the difference between these two and i want to know how it works

 

[Metal Messiah.]

Simply put, first of all, as we know, DLSS takes images rendered at a low resolution, then uses AI to fill in the blanks to make an image at a higher resolution, and it does so with very little reduction in image quality (give or take).

But, Frame Generation/FG, on the other hand, also referred to as DLSS 3.5, uses AI to upscale existing frames to a higher resolution. Frame Generation helps in creating entirely new frames using AI.

FG works by analyzing consecutive frames of a game, predicting the next frame, and then inserting a new, AI-generated frame between them.

So basically, FG can generate entirely new frames by predicting where each pixel in a scene is going to be moving in a given frame, and then using that information to generate what filler frames in between frames the GPU renders in a traditional way. This doesn’t come without any drawback though.

Because the Tensor Cores are guessing what’s going to be happening in the frames they’re generating, errors can happen at lower frame rates, where change is more apparent from frame to frame.

However, Frame Generation can also help bypass CPU bottlenecks.

You can think of FG as follows.

• Initial Frame Analysis: DLSS 3 analyzes two consecutive frames, to get an idea about what changes are occurring in the scene.
• Prediction done using AI: Then using a deep learning model, it predicts what the intermediate frame should look like, based on the motion observed.
• Frame Insertion: Here the AI-generated frame is inserted between the original frames.

 

[Metal Messiah.]

Btw, I have been using a lot of upscaling tech in my games, DLSS with FG, XESS, FSR, and when it comes to frame gen, I have noticed a lot of latency and input lag.

In heavy CPU-bound games not so much though. But definitely not worth it. I did get some extra FPS by using these upscaling tech, but that didn't FEEL like it.

It was more of a "placebo" effect for me.

 

[ridwan123]

Simply put, first of all, as we know, DLSS takes images rendered at a low resolution, then uses AI to fill in the blanks to make an image at a higher resolution, and it does so with very little reduction in image quality (give or take).

But, Frame Generation/FG, on the other hand, also referred to as DLSS 3.5, uses AI to upscale existing frames to a higher resolution. Frame Generation helps in creating entirely new frames using AI.

FG works by analyzing consecutive frames of a game, predicting the next frame, and then inserting a new, AI-generated frame between them.

So basically, FG can generate entirely new frames by predicting where each pixel in a scene is going to be moving in a given frame, and then using that information to generate what filler frames in between frames the GPU renders in a traditional way. This doesn’t come without any drawback though.

Because the Tensor Cores are guessing what’s going to be happening in the frames they’re generating, errors can happen at lower frame rates, where change is more apparent from frame to frame.

However, Frame Generation can also help bypass CPU bottlenecks.

You can think of FG as follows.

• Initial Frame Analysis: DLSS 3 analyzes two consecutive frames, to get an idea about what changes are occurring in the scene.
• Prediction done using AI: Then using a deep learning model, it predicts what the intermediate frame should look like, based on the motion observed.
• Frame Insertion: Here the AI-generated frame is inserted between the original frames.

Does frame generation render an image at low resolution and then upscale it to higher resolution?

 

[ridwan123]

Btw, I have been using a lot of upscaling tech in my games, DLSS with FG, XESS, FSR, and when it comes to frame gen, I have noticed a lot of latency and input lag.

In heavy CPU-bound games not so much though. But definitely not worth it. I did get some extra FPS by using these upscaling tech, but that didn't FEEL like it.

It was more of a "placebo" effect for me.

I also did not like DLSS because it let us to play in lower resolution not in our choosen resolution.i like to play pure and somehow it did not reach desire fps

 

[Moonstick2]

DLSS takes images rendered at a low resolution, then uses AI to fill in the blanks to make an image at a higher resolution...But, Frame Generation/FG, on the other hand, also referred to as DLSS 3.5, uses AI to upscale existing frames to a higher resolution.

It's a bit confusing as you seem to have said the same thing twice. Also I'm not sure the 3.5 thing is correct, but I don't have an Nvidia card so my DLSS knowledge is second hand.

My understanding is:
- DLSS uses AI to upscale rendered frames to a higher resolution.
- DLSS3 introduced Frame Generation, which uses AI to create new frames between the upscaled frames in the way you describe.
- DLSS3.5 added improvements to the AI upscaling for ray tracing.

 

[thestryker]

DLSS 3 is just Frame Generation for 40 series and undoubtedly future GPUs.

DLSS 3.5's main feature is Ray Reconstruction which is an AI enhanced denoiser to hopefully provide better image quality with ray tracing this doesn't have a requirement other than a tensor cores. If you're interested in some more details about 3.5: https://www.techspot.com/article/2740-dlss-35-ray-reconstruction/

I'm not 100% sure what exactly nvidia is calling DLSS now but that's just the upscaling technology.

Frame Generation will likely be an important technology for high refresh panels, but unless you're getting good baseline performance already it's not beneficial.

 

[Metal Messiah.]

Does frame generation render an image at low resolution and then upscale it to higher resolution?

NO.