It's hard to say because some games are explicit about it with an actual option, others just do it because they can. Also a cursory glance at the option I was looking into for Call of Duty ("fill remaining memory") may actually cause issues, but it's hard to say if this caching causing a problem. However there's another option called "Shader Preload" which reportedly helps reduce stuttering.
There was also a test I did with GTA V to study VRAM consumption behavior and found that with all of the VRAM consuming options turned down, i.e, the game settings reported the least amount it'd say the game would consume, it ended up consuming significantly more VRAM after running the benchmark. You can't really call this a "memory leak", because repeated tests afterwards don't cause VRAM consumption to increase. So I can only guess that GTAV will happily keep things in VRAM if there's enough space to do so.
Sometimes you're given no choice in the matter. Just because the option isn't there to toggle it doesn't mean the game isn't actually doing it. And besides, caching results
is a common way to speed up calculation-heavy workloads