Depends on which CPU you got and no there is no way that a GPU will do some tasks a CPU usually do.
If you have a high end GPU but a very bad CPU, the GPU will not be able to run at it's normal speed, because it will be bottlenecked by the CPU.
For example: Good GPU + Good CPU
100% GPU load, X% CPU load (totally fine)
Good GPU + Bad CPU
50% GPU load, 100% CPU load (bottlenecked)