When looking at the same generation, the lower cards have sections disabled. So it has less shaders, less render units, less texture units, just all around less resources to work with. Vram is separate from the gpu and they add an appropriate amount compared to the card's performance or needs. More vram does not necessarily mean better performance. When looking at the same model, clocks can be different. Beyond the same gen, there's a lot that goes into what affects performance and you can read books and books on those topics.
This pic is the 980ti. Compared to the titan x which has the full gm200 enabled. Fyi they laser cut gpus so can't be enabled.