What is the first upgrade I should make to my system?

It the card is worth it for Nvidia Physx, that's your call. BL2 has Physx and it looks pretty, but hardly any games use it.

What I would upgrade in that computer? The power supply, which I am sure is not 80+ certified. I would upgrade that before putting a better card in it.