The PSU is probably fine. Remember the motherboard has voltage regulators that stabilize the 12V source anyway.
CPU's are unlikely to "fry" as they have a thermal shut-down diode for worst-case but should auto-throttle down before that anyway if they overheat.
*Plenty of guides, but between 4.5GHz and 4.8GHz seems doable depending on the silicon lottery (your CPU) and your CPU cooler.
You'll want at least a Cryorig H7 Ultimate for overclocking (4.6GHz?), or liquid cooler (i.e. 2x140mm).
**75degC is what Intel recommends for optimal life of the PC (you can go over, but preferably not very often).
You can use CPUID's Hardware monitor for the core temps then adjust overclock based on that (I'd use HANDBRAKE to convert a video as a...