Worth it? Well, maybe, sometimes. Why do you want to OC the card? What do you hope to achieve.
You can install MSi Afterburner and run the OC (Overclock scanner). This will test your GPU and increase clockspeed or mem speed.
However, it is very conservative. You can also OC the GPU manually of course. And this gives best results. With that said, keep in mind that it doesn't bring that much of an uplift often. If you get a 5% performance boost for the core clocks, that may just translate to 1-5 FPS in game increase. Sometimes it's not worth it.
nVidia made it easy though. It uses an algorithm to determine thermal headroom, and voltage. It essentially boosts your GPU to it's highest performance and adjusts it on the fly.
Now, there are also other ways to OC a GPU. You do this with a vbios update. This can bring good results, but is not always guaranteed. You could conceivably brick the card.
The biggest option for overclocking Nvidia cards is COOLING them.
Use Afterburner.
Set a custom fan curve first and foremost.
I use 40c=40% fan speed to 80c=80% fan speed and anything over 80=100% fan speed.
Core clock max is already set in bios/drivers no way to get around it without special bios.
My 1060 and 2 1070s all capped out between 1950-2000mhz.
My 3060ti does 2040-2070mhz.
My 4070 does 2900-2970 mhz.
Temperature is the most limiting factor you have easy control of.
all of the above cards run folding@ home all of their life at those speeds. Until I retired them and gave them to friends/family. Still running 3060ti FE and 4070 Asus Dual.
Memory is ECC so when overclocking do it in small steps and check performance.
First small dip you get in performance benchmarks means you are getting correctable errors and any higher clock speed will not increase performance. So go back to you last lower setting.
Have fun tinkering.