A comprehensive study on fault tolerance in stream processing systems

Stream processing has emerged as a useful techno-logy for applications which require continuous and low latency computation on infinite streaming data.Since stream proces-sing systems(SPSs)usually require distributed deployment on clusters of servers in face of large-scale of data,it is especially common to meet with failures of processing nodes or communi-cation networks,but should be handled seriously considering service quality.A failed system may produce wrong results or become unavailable,resulting in a decline in user experience or even significant financial loss.Hence,a large amount of fault tolerance approaches have been proposed for SPSs.These appr-oaches often have their own priorities on specific performance concerns,e.g.,runtime overhead and recovery efficiency.Never-theless,there is a lack of a systematic overview and classifica-tion of the state-of-the-art fault tolerance approaches in SPSs,which will become an obstacle for the development of SPSs.Therefore,we investigate the existing achievements and develop a taxonomy of the fault tolerance in SPSs.Further-more,we propose an evaluation framework tailored for fault tolerance,demonstrate the experimental results on two represen-tative open-sourced SPSs and exposit the possible disadvanta-ges in current designs.Finally,we specify future research direc-tions in this domain.