Supercomputers as a Continous Medium
arxiv(2024)
摘要
As supercomputers' complexity has grown, the traditional boundaries between
processor, memory, network, and accelerators have blurred, making a homogeneous
computer model, in which the overall computer system is modeled as a continuous
medium with homogeneously distributed computational power, memory, and data
movement transfer capabilities, an intriguing and powerful abstraction. By
applying a homogeneous computer model to algorithms with a given I/O
complexity, we recover from first principles, other discrete computer models,
such as the roofline model, parallel computing laws, such as Amdahl's and
Gustafson's laws, and phenomenological observations, such as super-linear
speedup. One of the homogeneous computer model's distinctive advantages is the
capability of directly linking the performance limits of an application to the
physical properties of a classical computer system. Applying the homogeneous
computer model to supercomputers, such as Frontier, Fugaku, and the Nvidia DGX
GH200, shows that applications, such as Conjugate Gradient (CG) and Fast
Fourier Transforms (FFT), are rapidly approaching the fundamental classical
computational limits, where the performance of even denser systems in terms of
compute and memory are fundamentally limited by the speed of light.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要