Benchmarking cloth manipulation using action graphs: an example in placing flat

Benchmarking robotic manipulation is complex due to the difficulty in reproducing and comparing results across different embodiments and scenarios. Cloth manipulation presents additional challenges due to the complex object configuration space. Traditional cloth manipulation papers do not have well defined metrics to evaluate the success of a task or the quality of the result, and are tailored to each evaluation. In this paper we propose to evaluate cloth manipulation segmenting a task into steps that can be evaluated independently, and to study how their success measures influence in the next segment and relate to task as a whole. In particular, we study a popular task such as placing a cloth flat on a table. We propose a benchmark with simple but continuous evaluation metrics that explore the influence of grasp location into the quality of the task. Our results show that grasp location doesn’t need to be precise on corners, that quality measures focused on evaluating different cloth parts can enlighten issues to solve and that success definition of a segment has to consider its influence on the ability to perform successfully the next segment of action.