A-SOiD, an Active Learning Platform for Expert-Guided, Data Efficient Discovery of Behavior

To identify and extract naturalistic behavior, two schools of methods have become popular: supervised and unsupervised. Each approach carries its own strengths and weaknesses, which the user must weigh in on their decision. Here, a new active learning platform, A-SOiD, blends these strengths and, in doing so, overcomes several of their inherent drawbacks. A-SOiD iteratively learns user-defined groups and can considerably reduce the necessary training data while attaining expansive classification through directed unsupervised classification. In socially-interacting mice, A-SOiD outperformed other methods and required 85% less training data than was available. Additionally, it isolated two additional ethologically-distinct mouse interactions via unsupervised classification. Similar performance and efficiency were observed using non-human primate 3D pose data. In both cases, the transparency in A-SOiD’s cluster definitions revealed the defining features of the supervised classification through a game-theoretic approach. Lastly, we show the potential of A-SOiD to segment a large and rich variety of human social and single-person behaviors with 3D position keypoints. To facilitate use, A-SOiD comes as an intuitive, open-source interface for efficient segmentation of user-defined behaviors and discovered sub-actions.