The VRAM type will not affect whether or not it works.
As hotaru.hino rightly pointed out, your PSU has to be able to handle the increased power demand. The GTX 950 has a TDP of 90W while the GTX 1080 Ti has a TDP of 250W. That's a 160W difference which is huge.