An reinforcement learning approach for allocating software resources

Software resource allocation is an significant factor of system configuration which plays a critical role in guaranteeing the performance of multitier web service systems. Computing the optimal allocation of different software resources in order to meet performance requirements under dynamic workloads conditions is in highly challenging. Existing approaches mostly rely on translating domain knowledge from experts into computational solutions through heuristics-based optimization techniques. While such techniques are useful, they cannot leverage actual usage data generated by system users which may contain allocation strategies that are not captured by domain experts' knowledge. In this paper, we propose an iterative feedback mechanism which solves the problem to some extent by optimizing software resource allocation of multitier web systems through imitating system users who have achieved excellent performance. Specifically, we propose a deep Q-learning network-based approach for performance prediction to deal with the dynamic changes of complex workloads. The performance prediction method involves the reinforcement learning method for capturing the dynamics of online software resource allocation, and then computing the current optimal policy. We implement the approach in the multitier web benchmark system, and the experimental results demonstrated significant improvement compared to models built based on domain knowledge.