Tech Report # 2019-05 : Performance Interference of Big Data Frameworks in Resource Constrained Clouds

Stratos Dimopoulos, Chandra Krintz, Rich Wolski

semanticscholar(2019)

引用 0|浏览0
暂无评分
摘要
Correspondence: stratos@cs.ucsb.edu Department of Computer Science, University of California, Santa Barbara, USA Full list of author information is available at the end of the article Abstract In this paper, we investigate and characterize the behavior of “big” and “fast” data analysis frameworks, in multi-tenant, shared settings for which computing resources (CPU and memory) are limited. Such settings and frameworks are frequently employed in both public and private cloud deployments. Resource constraints stem from both physical limitations (private clouds) and what the user is willing to pay (public clouds). Because of these constraints, users increasingly attempt to maximize resource utilization and sharing in these settings. To understand how popular analytics frameworks behave and interfere with each other under such constraints, we investigate the use of Mesos to provide fair resource sharing for resource constrained private cloud systems. We empirically evaluate such systems using Hadoop, Spark, and Storm multi-tenant workloads. Our results show that in constrained environments, there is significant performance interference that manifests in multiple ways. First, Mesos is unable to achieve fair resource sharing for many configurations. Moreover, application performance over competing frameworks depends on Mesos offer order and is highly variable. Finally, we find that resource allocation among tenants that employ coarse-grained and fine-grained framework scheduling, can lead to a form of deadlock for fine-grained frameworks and underutilization of system resources.
更多
查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要