Yes, you can adjust the power limits in Afterburner, but you can't just keep increasing it to point of failure. On Nvidia cards particularly you are usually limited to something like 115% on the top end. And that depends on the base number the vBIOS has in it, so sometimes you will see 108%, but that is because the card has a higher preset floor. The ceiling is defined by Nvidia at the factory. You need modified vBIOS to exceed these values. or do shunt mods.
Pretty much since Maxwell you have been unable to modify voltage beyond some token increases, and even then, it may only have been visual and not actually changed the card. On newer cards, they don't even show you voltage, only power.
Typical Nvidia overclocking today:
Max power limit
Max temperature limit
Max fans (or what you consider tolerable)
Add boost until it becomes unstable, note
Add memory boost until it becomes unstable, note
Try your two maximums together, if it crashes, reduce one or the other. But one at a time. Compare some benchmark results online, while a card may appear stable, recoverable memory errors can occur and reduce performance. (GDDR6 and up also have ECC built in, so they don't tend to crash, just go really slow)
Once you have confirmed the GPU is stable you have reached maximum performance.
As always, monitor temperatures and keep it below 90C or so. late model Nvidia likes to be as cool as possible though and the default throttle point on many cards is 83C, though you will also see 73C on some cards. These usually have beefy coolers and they were aiming for silence.
Just remember you are only really messing with a boost target, the actual results will generally not be that. So it will bounce around your target, and probably never reach it.