Measuring Contribution in Collaborative Writing: an Adaptive NMF Topic Modelling Approach

In universities worldwide, instructors may spend a significant amount of time reviewing homework and group projects submitted by their students. Web-based technologies, like Google Docs, have provided a platform for students to write documents collaboratively. Currently, those platforms provide limited information on the individual contribution made by each student. Previous studies have focused on the quantitative aspects of individuals' contribution in collaborative writing, while the quality aspect has received less attention. In this paper, we propose a new model to measure not only quantitative input but also the quality of the content that has been contributed to a document written collaboratively in Spanish language. Based on topics-modeling techniques, we use an adaptive non-negative matrix factorization (NMF) model to extract topics from the content of the document, and grade higher students making those contributions. Using Google documents submitted by students to the academic system of our university as part of their projects, experimental results show that compared to other baseline methods such as edits or words count, our model provide a better approximation to the scores given by human reviewers. Therefore, our model can be used as part of an automatic grading subsystem within the academic system, to provide a baseline score of students' contribution in collaborative documents. This will allow instructors to reduce their workload associated with revision and grading of documents and focus their time on more relevant tasks.