If you haven't already, use DDU to uninstall all previous drivers and then reinstall AMD drivers. If that doesn't change anything, look at CPU usage, you may be bottlenecked. Another way to check for this is to lower resolution and see if FPS improves significantly. If not, it's probably a CPU bottleneck.
I just tried that and there is a little bit of improvement in some games, but Gta V for example used to ran at 60fps everything on high, now it is at 30fps.
Is Battlefront 2 one of the games where you get significantly worse performance than with the GTX 1050 with the same settings? If so, can you verify that the CPU frequency maintains a consistent ~3.7-4 GHz? And that the GPU maintains a consistent clock speed roughly around 1200 MHz?
I don't really trust task manager. Try something like HWiNFO64. Looks at the highest CPU usage across the individual cores. But like I said, I think you should look at CPU and GPU frequency to make sure nothing is throttling.
So you're getting significantly better FPS with the 570... so what exactly is the problem? If you're only having issues with certain games, then the results for those games are what's relevant.
Are you using 1x8 gn stick or ram in single channel or or 2 sticks of ram in dual channel? Running a single stick becomes an issue when you're running faster hardware.