Yes, you need to install the driver. I'd also recommend that you turn off the Windows 10 automatic driver updates. They just seem to cause issues with graphics cards in particular.
For starters, that's a particularly bad power supply. Chieftec power supplies are extremely low quality, and it wouldn't be surprising if a bad power supply was the source of your troubles in the beginning, rather than your other graphics card.
I would however run the Display driver uninstaller and then install the latest driver for your card.
DDU is here:
http://www.guru3d.com/files-get/display-driver-uninstaller-download,20.html
Current driver is here:
http://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/Windows/364.51/364.51-desktop-win10-64bit-international-whql.exe&lang=us&type=GeForce
And full instructions on running the DDU and installing the drivers are here:
http://www.tomshardware.com/faq/id-2767677/clean-graphics-driver-install-windows.html
I would also get a different power supply. If you recently bought that one, I'd see if I could return it. Sorry, but it's junk. Not fit for use with a discreet graphics card. It has plenty of capacity, going off the label, unfortunately the label can't tell you how poor the internal quality is and that it's very unlikely it can supply that labeled capacity or even run under the demand of a discreet graphics card.
It could be however that the majority of your problem IS only driver related, so I'd do that first, if the PSU allows you to without crashing the card.
You might have to put the old card back in, run the DDU including the safe mode reboot and then a restart, and then install the new drivers, then shut down, remove the old card, install the new card and restart.