don't get me wrong, that is a decent pc, old but decent, small, not noisy and doesn't get hot, right?
but dell was only thinking on internet explorer, excell, word and lots of accounting apps when made their pcs
one question first, your dell looks like this?
or like this
the first one is the sff, small form factor, the second is the normal one, the normal one has space for both soundcard and gpu, the sff only has space for the soundcard, pci, not pci express, and a gpu but should be low profile single slot gpu, so not a gaming capable gpu, only old quadros would fit in there, and not much in small form factor low profile shape
for the normal size one, if you disable the onboard soundcard and put a xonar soundcard, things should change alot, like day and night difference
if you are not going to use hdmi, then buy any gpu you can fit in there, low profile gpu must be, so a gtx 750 or 1050 low profile is what you need, there is some amd but really a bit hard to find
i would buy one like this
https://www.newegg.com/Product/Product.aspx?Item=N82E16829132020
not sure if in the box comes the low profile bracket, probably comes
there are alot of better sound card out there but this one should be the starting point
https://www.newegg.com/Product/Product.aspx?Item=N82E16814137079&cm_re=gtx_1050_low_profile-_-14-137-079-_-Product
that will make the old dell into a gaming pc, that is based on the idea that the psu will handle the gtx, something that should, usually those come with a 300 watts psu, if it doesn't have a psu this big, you can buy a 430 watts psu for this particular pc
for the sff, well, i wouldn't change anything, heat, psu size and cost of low profile parts would make any modification too complicated, perhaps the soundcard only