Active learning for sketch recognition

The increasing availability of pen-based tablets, and pen-based interfaces opened the avenue for computer graphics applications that can utilize sketch recognition technologies for natural interaction. This has led to an increasing interest in sketch recognition algorithms within the computer graphics community. However, a key problem getting in the way of building accurate sketch recognizers has been the necessity of creating large amounts of annotated training data. Several authors have attempted to address this issue by creating synthetic data, or by building easy-to-use annotation tools. In this paper, we take a different approach, and demonstrate that the active learning technology can be used to reduce the amount of manual annotation required to achieve a target recognition accuracy. In particular, we show that by annotating few, but carefully selected examples, we can surpass accuracies achievable with equal number of arbitrarily selected examples. This work is the first comprehensive study on the use of active learning for sketch recognition. We present results of extensive analyses and show that the utility of active learning depends on a number of practical factors that require careful consideration. These factors include the choices of informativeness measures, batch selection strategies, seed size, and domain-specific factors such as feature representation and the choice of database. Our results imply that the Margin based informativeness measure consistently outperforms other measures. We also show that active learning brings definitive advantages in challenging databases when accompanied with powerful feature representations. Graphical abstractDisplay Omitted HighlightsWe present a set of carefully designed experiments and a battery of accompanying statistical tests, which will serve as a roadmap to follow for practitioners of active learning who wish to perform factor analysis.We present the first extensive empirical analysis on active learning for sketch recognition, and provide a detailed discussion of the analysis results.We determine the best performing and reliable informativeness measure for sketch recognition.We show that starting with a large seed set yields better active learning performance for the single classifier approach.We show that the use of active learning brings definitive advantages in challenging databases when accompanied with powerful feature representations.