News US Prohibits Exports of Nvidia's A800 and H800 to China, Blacklists Chinese GPU Developers

[Admin]

The U.S. will now prohibit the export of Nvidia's A800 and H800 to China, and has added Biren Technology and Moore Threads to its Entity List to prohibit them from receiving certain exports. This is being done to slow down China's AI development.

US Prohibits Exports of Nvidia's A800 and H800 to China, Blacklists Chinese GPU Developers : Read more

 

[jp7189]

At what point does supercomputer scaling hit the wall of diminishing returns? If one has unlimited space, power, cooling can more nodes be added to make up for lower density performance?

 

[InvalidError]

At what point does supercomputer scaling hit the wall of diminishing returns? If one has unlimited space, power, cooling can more nodes be added to make up for lower density performance?

Supercomputer scaling is ultimately limited by inter-process communications. The more tightly integrated the whole supercomputer is, the easier it is to minimize IPC bottlenecks. If your software wastes 1% of its time on IPC overhead, your practical scaling is limited to about 100 CPUs. Get that down to 0.01% and you should be able to achieve gains up to 10 000 CPUs. If you want to make meaningful use of even more CPUs than that, you need to work even harder on reducing IPC overhead.

You can probably imagine how slashing overhead from 1% to 0.1% should be considerably easier than going from 0.01% to 0.001% if you want your algorithm to scale to 100k cores. Going further requires optimizing almost no IPC left to even less.