In 90% of cases, more money gets you better product. Other 10% is branding, Beats headphones for example are often no better than ones half the cost. Bose also, there are often cheaper and better sounding things. Cables is another one, a $5 HDMI cable does the job as good as a $30 one.
What you get depends on what you like, if you never used a good keyboard or mouse, it's tough to say that you will actually prefer them or see a difference.
If you have a BestBuy near you, go there, and try some things out. With so many computer stores shut down now, I find that BestBuy actually has the best selection of gaming keyboards to try out, and standard ones also. You can even find some good prices there when things go on sale, last two mechanical keyboards and my Bose bluetooth headphones (yes BOSE LOL, but their bluetooth headsets are actually pretty good even for the money) I got cheaper than they were form Amazon or Newegg.
A $50-70 headset is a good price point for good quality and price, HyperX Cloud Stinger or Steelseries Acrtics 3 are probably the best for the price for a lower end setup.
For keyboards and mice there are so many that it can be hard to pick without trying them.
On the low end scale, I liked the Cooler Master Devastator combo, my son used that for a bit, and I used it a few times, felt pretty good.