By the same logic one would think the A770 8GB would be 14% faster than the A750, but it is less than 7%. Usually shader performance doesn't tell the whole story.
The B580 has 20% higher memory bandwidth, 11% more shaders, no more ROPS and is held back more by driver overhead. Also that reduced power limit on the B570 will only matter in a few scenarios like benchmarks. In games that 150W (x1.2=180W if it has the same % extra as the B580) is not going to hold it back. And that clock speed difference is artificial. You should be able to get similar clocks out of both unless there is some hard lock.
I'd say wait for game benchmarks. It may vary per game.
Also Tech Yes City tested the B580 with a 10400T vs a 9800X3D and the 9800X3D was only 11% faster at 1080p. A 10400T running like 3.2GHZ with 2666c16. The RX7600 lost 10% in the same test. That 5600 looks really slow now. And maybe my 13900kf is overkill.