Yes, there's no requirement to update a GPU driver if the one installed is working fine for you. Most newer drivers target newer games with optimization.
However, if using associated software, part of the driver package (like Geforce Experience), it would be prudent to review what's being addressed in driver updates. Recently, there's a GeForce Experience exploit fix within the driver update.... So if you use GeForce Experience, it would be a good idea to update the driver.