NVM-based energy and cost efficient HPC clusters

Non-Volatile Memory (NVM) is a byte-addressable, high capacity, high latency and a persistent form of memory that can extend the primary memory hierarchy by another level. It is up to 8x denser than DRAM, allowing for clusters with compute nodes that have significantly higher memory capacity than those of previous generations. Intel’s Optane DC Persistent Memory Module (PMM) is such an NVM device that can be used to increase the memory density of high performance computing (HPC) systems. This work hypothesizes that with highermemory density, scientific computing applications with larger problem sizes can be run on fewer compute nodes than on current HPC systems. This, in turn, can reduce operational cost. This work tests this hypothesis by comparing performance and energy of HPC jobs with large problem sizes of (1) fewer nodes with large NVM capacity under various configurations and (2) more nodes with an equivalent amount of DRAM memory. In experiments, performance and energy consumption are shown to be dependent on application characteristics: Codes optimized for high cache reuse suffer no performance degradation on NVM, combined with significant energy savings at lower acquisition and operational costs compared to traditional HPC systems without NVM. In contrast, memory bound applications using DRAM as a cache for NVMmay provide a small performance benefit over using a DRAM-NVM hybrid memory.