The main news seems to be fp64 and PCIe4.
Deep learning performance is almost a footnote. It does seem that AMD was blindsided by Nvidia's Tensor cores. That said, at least they have respectable fp64 performance to fall back on (for those HPC applications requiring it).
Anyway, I'm really suspecting 4096 is some kind of architectural ceiling to the shader count, imposed by GCN. They first reached that with 28 nm Fury, back in 2015, and have never gone beyond. This has really got to be hurting, since there's only so much you can do with clock speed. That said, on such a new process like 7 nm, perhaps it wouldn't make a lot of sense to try and go bigger.
Eh, color me disappointed. I knew it wasn't going to take back the crown, but I was hoping for a little more improvement over first-gen Vega. Maybe something that could challenge a GTX 1080 Ti.