From what I read you may want to add the GPU into your HP PC.
So
1) Can you post what GPU you want? if you want to add the GPU.
2) Where do you shop the PSU?
3) What budget do you have?
4) Check your MB to see it had the newer version BIOS or not, also try to find the BIOS download from HP, in case you need it.
And before you buy the PSU and GPU, try this first: borrow one, or if you have one GPU, then try to use the GPU to see you can boot the PC or not, also make sure the GPU will fit into your HP PC too.
HP Desktop PCs - Computer Does not Start After Installing a Video
https://support.hp.com/us-en/document/c03653200