Considering the A100 has 40/80GB models, its not too bad although its never enough. As for price, no doubt they cost more, but these aren't meant for consumer cards so don't it doesn't matter that much as well. Performance matters more. The A100 cost over 10K.How about the Capacity and Price per GB ? this is the most important factor here . bandwidth is more than enough for today hardware.
I can see AMD using HBM3 for their Professional line of cards and for a future dedicated mobile line of cards.I have been wondering for a few years now whether AMD will release a CPU with with their I/O die, chiplets and a HBM chip.
Yes there would need to be a significant redesign of the I/O die, substrate, power delivery and how the caching works, but this could be a very powerful option.
I would expect the HBM to either be used as a victim cache for the L3, or as the first part of main memory, but I expect that would cause problems (the RAM equivalent of scheduler issues.?)
Of course the best IMHO use case for this would be for the above configuration but with one of the chiplets replaced with a competent GPU for a very powerful APU. This however seems to be wasteful as the HBM would cost a load of money for its capacity, and would that capacity be enough to not be a bottleneck... Probably not.
The other option I foresee would be for AMD to make a monolithic die (as it currently does) but with the Memory interface expanded to use HBM as well and to use this as dedicated RAM for the GPU, again, this seems to be wasteful and would still encounter other problems leaving this idea to be better suited for CPU usage.
Such an idea might work very well for AMD server CPU's, but the added complexity might outweigh the gains, still, an interesting thought experiment.
Note that I have not mentioned Intel, simply because it does not have any current chiplet based products which this would IMHO be best suited for, but also because this is not a long way off of Intel's various Foveros options anyway.
Both AMD and Intel would have already gone through all of these options and looked at whether or not it would work effectively, be cost, power and performance effective as well as the manufacturing hurdles. "IF" such a thing is in the works, I would expect it to be baked into the design from the beginning and that would likely mean that "if" it ever sees the light of day it will be Zen 5 at the earliest IMHO.
"IF" and any such design using HBM as well as DDR5 as an option wins out, it would also have to compete against 3D stacked cache (which AMD demoed, but could be 4x higher capacity with a greatly higher cost) and 3D stacked CPU dies, the industry is incredibly interesting right now.
Apple and Ampere could always do such a thing, and IMHO it would be relatively easy for Ampere to do so because they are now bringing their designs in house and the only produce one CPU (with different SKU's).
Mods, please let me know if this needs to be a post on its own rather than here in this thread.