If the pcie 4.0 is utilized fully on all lanes, and the device could take benefit because higher transfer rate, then it pretty much better than pcie 3.0.
Do you need it? depends on the device that uses pcie 4.0, if it's x4 nvme ssd, or even a AIC ssd on raid 0 with pcie 4.0, it will be pretty much benefitable since it's double the bandwith over x4 on pcie 3.0.
For gpu, pcie 4.0 could be the next base standard, but for now there isn't any much benefit for pcie 4.0, but it's not wrong to invest in it. The same as DDR3 and DDR4, it got double the bandwith of ddr3, but if you don't feel the need to spend more on newer gen, you could stay with older gen as long as it's not too old (heck im using ddr3 and it's pretty fine for now).
While agree it depends on the device, it also depends on the application.
For M.2 NVME's drives...Windows doesn't function on long sequential reads, but small random reads. PCIe 4 only helps large sequential reads so the benefit to Window's operation is seen with just about any decent SSD. Where it would help is moving large multi-GB files from one NVME drive to another with huge DRAM cache. Most people just don't do that so much and if you did you'd need both to be PCIe gen 4.
It can help long data reads to GPU's but that doesn't seem to benefit gaming much.
A very telling point about PCIE 4.0 for gpu’s is when NVIDIA did their launch of the 3090 they used a system running 3.0 for the benchmarks. If there had been any benefit they would have used a 4.0 setup to fully showcase the gpu but it makes no difference. What and when it will make a difference is unknown, maybe next generation but then maybe not.