The case for tiny tasks in compute clusters
HotOS, pp. 14-14, 2013.
data-parallel jobsmall tasksmaller taskshort tasktask execution frameworkMore(5+)
We argue for breaking data-parallel jobs in compute clusters into tiny tasks that each complete in hundreds of milliseconds. Tiny tasks avoid the need for complex skew mitigation techniques: by breaking a large job into millions of tiny tasks, work will be evenly spread over available resources by the scheduler. Furthermore, tiny tasks al...More
PPT (Upload PPT)