Fuzzy adaptive control for heterogeneous tasks in high-performance storage systems

Beyond handling user reads and writes, storage systems execute multiple background tasks of various types, such as reconstruction of missing parity data and defragmentation. The resources of the system must be divided between user loads and internal tasks using a specific policy. This work describes Fuzzy Adaptive Control -- an innovative mechanism for sharing resources among various types of highly-variable loads. The new approach uses throughput as the task progress indicator avoiding assumptions about task properties such as resource consumption or handling process. It makes this new technique particularly well suited for complex systems where defining an accurate task model is difficult. The Fuzzy Adaptive Control is evaluated on resource division between user loads and background tasks in HYDRAstor -- a commercial high-performance distributed secondary storage system. The presented mechanism is compared with fair queuing variants and is shown to be more stable in the case of irregular workload. The evaluation proves that our approach is responsive to changing load conditions and ensures high resource utilization.