LearnedWMP: Workload Memory Prediction Using Distribution of Query Templates
CoRR(2024)
摘要
In a modern DBMS, working memory is frequently the limiting factor when
processing in-memory analytic query operations such as joins, sorting, and
aggregation. Existing resource estimation approaches for a DBMS estimate the
resource consumption of a query by computing an estimate of each individual
database operator in the query execution plan. Such an approach is slow and
error-prone as it relies upon simplifying assumptions, such as uniformity and
independence of the underlying data. Additionally, the existing approach
focuses on individual queries separately and does not factor in other queries
in the workload that may be executed concurrently. In this research, we are
interested in query performance optimization under concurrent execution of a
batch of queries (a workload). Specifically, we focus on predicting the memory
demand for a workload rather than providing separate estimates for each query
within it. We introduce the problem of workload memory prediction and formalize
it as a distribution regression problem. We propose Learned Workload Memory
Prediction (LearnedWMP) to improve and simplify estimating the working memory
demands of workloads. Through a comprehensive experimental evaluation, we show
that LearnedWMP reduces the memory estimation error of the
state-of-the-practice method by up to 47.6
single-query model, during training and inferencing, the LearnedWMP model and
its variants were 3x to 10x faster. Moreover, LearnedWMP-based models were at
least 50
advantages of the LearnedWMP approach and its potential for a broader impact on
query performance optimization.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要