CLOUDFACTORY: An Open Toolkit to Generate Production-like Workloads for Cloud Infrastructures

Cloud infrastructures are large-scale and complex platforms designed to host a wide diversity of applications and workloads. Given these complexity and scale factors, simulators and benchmarks are broadly adopted in vitro to study their behaviors, prototype new software components and heuristics, and evaluate their effective performances. However, both state-of-the-art simulations and benchmarks may suffer from a representativeness problem, as the reported results can vary depending on their input workloads. For example, a Infrastructure-as-a-Service (IAAS) platform aims to host Virtual Machines (VMs), whose characteristics (resource configurations, workload intensity, arrival/departure rate, etc.) can greatly differ depending on Cloud providers and public/private deployments. Addressing this IAAS representativeness thus requires Cloud providers to share production-scale datasets, which might be considered sensitive. Moreover, Simulations and benchmarks require a specific experiment scenario that cannot be easily generated from Cloud providers characteristics. To address these issues, this paper introduces CLOUDFACTORY, a IAAS workload generator. Our contribution is first composed of a library that can be used by Cloud providers to share IAAS statistics, instead of raw datasets. Then, we introduce a generator designed to produce realistic VM workloads that match these statistics. CLOUDFACTORY is made available as open-source software that can be adopted by Cloud providers and researchers to foster the evaluation of new contributions. As an example, we perform an analysis on scheduling evolution for different IAAS workload intensity of two different Cloud providers: Microsoft Azure and Chameleon. We also report on OVHcloud statistics computed from CLOUDFACTORY and compare them to other Cloud providers.