Does it have Sata II? If so I would recommend cloning the OS drive to an SSD - this will generate the single being performance improvement possible for this rig. Also put more RAM in if possible, and if it can be done inexpensively - 8GB would be ideal.
The standard GPU upgrade for an old PC is the Geforce GTX 750Ti, because of it's low power requirement - there are examples available that need no additional power connector - but check what the PSU is capable of. But read around as to how much gain that might generate, as I don't know whether that CPU and associated mono have the right slots etc, or might hold that GPU back
You might decide that you want to recommend a new PSU as part of the upgrade if this machine is going to...