Multimodal Scoring Model for Handwritten Chinese Essay.

Essay writing plays a critical role in Chinese language skill teaching. With the smart education becomes a hot topic, the demand for automatic essay scoring (AES) has been emerging among teachers and students. Existing works frequently ignore the impact of the visual modal during the scoring process, such as writing quality in terms of neatness or legibility. This paper addresses the problem with a visual-textual integrating perspective and proposes a deep learning based multi-modal AES. Specifically, implicit alignment algorithm is presented to cohere the distinct visual modal and text modal. Methods are tested on a large-scale dataset consisting of over 4000 essays including HSK publicly available samples. The results show that multi-modal AES reduce the MAE of scoring from 1.13 to 1.06, and the implicit alignment algorithm reduces it further to 1.01.