the more ram you have the more the OS will use, until it gets its fill. When you start a foreground application such as a game, generally the OS will cache background processes to the virtual memory.
So basically until you reach above 90% of ram usage you should not suffer performance loss during a game for example.