Can you install the 1060 on your current system /motherboard Sure it will run.
Will the GTX 1060 performance be affected by your motherboard and CPU, compared to the GTX 770? well it will be night and day while you game, and if you do have bottle necking it will be so minimal that you should not feel it in fps
the real question is, will the new video card a) fit in your case, b) will you power supply have the 6pin and 6+2 pins for power the video card needs.
so please answer these questions please
1) what Case is your hardware in ?
2) what is the make and model of your power supply.