Concurrency Management in Heterogeneous Architectures

Heterogeneous architectures consisting of generalpurpose CPUs and throughput-optimized GPUs are projected to be a dominant computing platform for many classes of applications. The design of such systems is more complex than for a homogeneous architecture because maximizing resource utilization while minimizing the interference between CPU and GPU applications is difficult. We show that GPU applications tend to monopolize the shared resources such as memory and network because of their high thread-level parallelism (TLP). To solve this problem, we propose an integrated concurrency management strategy that modulates the TLP in GPUs to control the performance of both CPU and GPU applications. It considers both GPU core state, and system-wide memory and network congestion information to dynamically decide on the level of GPU concurrency to maximize system performance. We propose two schemes, one targeted specifically for unilaterally boosting CPU performance (CM-CPU), and the other (CMBAL) for a balanced improvement of both CPU and GPU applications. We show that both of our schemes reduce the monopolization of the shared resources by GPU traffic. CMBAL also allows the user to control the performance tradeoff between CPU and GPU applications. To our knowledge, this is the first work that introduces new GPU concurrency management mechanisms to improve both CPU and GPU performance in heterogeneous systems.