It should just fit, assuming they are the same form factor. You will need to make sure to plug everything in correctly to the motherboard, but it should just be as simple as matching up the new cables with how the old ones were plugged in. One thing to check is if the PSU will have a PCIe power connector, preferable a 6+2 (total of 8 pin) one.
I'm pretty sure the PSU will have only a 6 pin PCIe power connector, which will be fine for a GTX 1650 Super (150-175 USD) however if you want the next size of up of GPU like the GTX 1660 Super (220-250 USD) you will either need to try and see if the card will run with just the 6 pin plugged into the 8 pin connector ( some will some wont) or buy a 6pin-8pin adapter cable.
These cables can be sketchy, so if you do buy one don't just buy the cheapest one you come across, take the time to read the reviews. Startech has a decent one for around 8 USD.
I would not go with any more powerful card with an HP PSU, and I would use a high draw card such as a RX580 or 590. The 1660 Super draws at most 130 watts, so that's a fairly safe margin of error for less then stellar PSU's