Yeah, that system is so old that ANY power supply that might have come in it would be incredibly suspect by now, but so would the motherboard and storage devices if they are the original parts.
Make sure you've done all of the relevant things below especially updating critical hardware drivers for the motherboard, from the motherboard product page, latest BIOS, latest Nvidia driver for that card, etc. If none of that works, then it's looking a lot more likely that you have a PSU or GPU card fault somewhere, possibly motherboard. All these things have capacitors which fail with time and age.
If there are any steps listed here that you have not already done, it would be advisable to do so if for no other reason than to be able to say you've already done it and eliminate that possibility.
Make sure your motherboard has the MOST recent BIOS version installed. If it does not, then update. This solves a high number of issues even in cases where the release that is newer than yours makes no mention of improving graphics card or other hardware compatibility. They do not list every change they have made when they post a new BIOS release. In cases where you DO already have the latest BIOS version, simply resetting the BIOS as follows has a fairly high percentage chance of effecting a positive change in some cases so it is ALWAYS worth TRYING, at the very least.
BIOS Hard Reset procedure
Power off the unit, switch the PSU off and unplug the PSU cord from either the wall or the power supply.
Remove the motherboard CMOS battery for about three to five minutes. In some cases it may be necessary to remove the graphics card to access the CMOS battery.
During that five minutes while the CMOS battery is out of the motherboard, press the power button on the case, continuously, for 15-30 seconds, in order to deplete any residual charge that might be present in the CMOS circuit. After the five minutes is up, reinstall the CMOS battery making sure to insert it with the correct side up just as it came out.
If you had to remove the graphics card you can now reinstall it, but remember to reconnect your power cables if there were any attached to it as well as your display cable.
Now, plug the power supply cable back in, switch the PSU back on and power up the system. It should display the POST screen and the options to enter CMOS/BIOS setup. Enter the bios setup program and reconfigure the boot settings for either the Windows boot manager or for legacy systems, the drive your OS is installed on if necessary.
Save settings and exit. If the system will POST and boot then you can move forward from there including going back into the bios and configuring any other custom settings you may need to configure such as Memory XMP, A-XMP or D.O.C.P profile settings, custom fan profile settings or other specific settings you may have previously had configured that were wiped out by resetting the CMOS.
In some cases it may be necessary when you go into the BIOS after a reset, to load the Optimal default or Default values and then save settings, to actually get the hardware tables to reset in the boot manager.
It is probably also worth mentioning that for anything that might require an attempt to DO a hard reset in the first place, IF the problem is related to a lack of video signal, it is a GOOD IDEA to try a different type of display as many systems will not work properly for some reason with displayport configurations. It is worth trying HDMI if you are having no display or lack of visual ability to enter the BIOS, or no signal messages.
Trying a different monitor as well, if possible, is also a good idea if there is a lack of display. It happens.
Go to the product page for your motherboard on the manufacturer website. Download and install the latest driver versions for the chipset, storage controllers, audio and network adapters. Do not skip installing a newer driver just because you think it is not relevant to the problem you are having. The drivers for one device can often affect ALL other devices and a questionable driver release can cause instability in the OS itself. They don't release new drivers just for fun. If there is a new driver release for a component, there is a good reason for it. The same goes for BIOS updates. When it comes to the chipset drivers, if your motherboard manufacturer lists a chipset driver that is newer than what the chipset developer (Intel or AMD, for our purposes) lists, then use that one. If Intel (Or AMD) shows a chipset driver version that is newer than what is available from the motherboard product page, then use that one. Always use the newest chipset driver that you can get and always use ONLY the chipset drivers available from either the motherboard manufacturer, AMD or Intel.
IF you have other hardware installed or attached to the system that are not a part of the systems covered by the motherboard drivers, then go to the support page for THAT component and check to see if there are newer drivers available for that as well. If there are, install them.
Make sure your memory is running at the correct advertised speed in the BIOS. This may require that you set the memory to run at the XMP profile settings. Also, make sure you have the memory installed in the correct slots and that they are running in dual channel which you can check by installing CPU-Z and checking the Memory and SPD tabs. For all modern motherboards that are dual channel memory architectures, from the last ten years at least, if you have two sticks installed they should be in the A2 (Called DDR4_1 on some boards) or B2 (Called DDR4_2 on some boards) which are ALWAYS the SECOND and FOURTH slots over from the CPU socket, counting TOWARDS the edge of the motherboard EXCEPT on boards that only have two memory slots total. In that case, if you have two modules it's not rocket science, but if you have only one, then install it in the A1 or DDR4_1 slot.
Fourth (And often tied for most important along with an up-to-date motherboard BIOS),
A clean install of the graphics card drivers. Regardless of whether you "already installed the newest drivers" for your graphics card or not, it is OFTEN a good idea to do a CLEAN install of the graphics card drivers. Just installing over the old drivers OR trying to use what Nvidia and AMD consider a clean install is not good enough and does not usually give the same result as using the Display Driver Uninstaller utility. This has a very high success rate and is always worth a shot.
If you have had both Nvidia and AMD cards installed at any point on that operating system then you will want to run the DDU twice. Once for the old card drivers (ie, Nvidia or AMD) and again for the currently installed graphics card drivers (ie, AMD or Nvidia). So if you had an Nvidia card at some point in the past, run it first for Nvidia and then after that is complete, run it again for AMD if you currently have an AMD card installed.
Graphics card driver CLEAN install guide using the Wagnard tools DDU
And last, but not least, if you have never done a CLEAN install of Windows, or have upgraded from an older version to Windows 10, or have been through several spring or fall major Windows updates, it might be a very good idea to consider doing a clean install of Windows if none of these other solutions has helped. IF you are using a Windows installation from a previous system and you didn't do a clean install of Windows after building the new system, then it's 99.99% likely that you NEED to do a CLEAN install before trying any other solutions.
How to do a CLEAN installation of Windows 10, the RIGHT way