What causes fps drop?
When a GPU runs out of VRAM, it will use RAM as part of it's memory, which is much slower and causes stuttering. But in your situation you should be fine since 8GB is more than enough even for 4K.
Also, no matter how much of data a GPU stores into it's VRAM module (let it be 4GB or 6GB), GPU also has to work with that data -> render images and send them to the screen. A graphics card must have enough horsepower to work with bigger data. If you max-out everything on a game and you have enough VRAM, the data will be stored in VRAM, but if the GPU does not have enough power to load/render/work with the amount of data it is given, the frames will take more time to be rendered, which means less frames/second and...