ParAlg: A Paraphasia Algorithm for Multinomial Classification of Picture Naming Errors

Purpose: A preliminary version of a paraphasia classification algorithm (hence- forth called ParAlg) has previously been shown to be a viable method for coding picture naming errors. The purpose of this study is to present an updated ver- sion of ParAlg, which uses multinomial classification, and comprehensively eval- uate its performance when using two different forms of transcribed input. Method: A subset of 11,999 archival responses produced on the Philadelphia Naming Test were classified into six cardinal paraphasia types using ParAlg under two transcription configurations: (a) using phonemic transcriptions for responses exclusively (phonemic-only) and (b) using phonemic transcriptions for nonlexical responses and orthographic transcriptions for lexical responses (ortho- graphic-lexical). Agreement was quantified by comparing ParAlg-generated para- phasia codes between configurations and relative to human-annotated codes using four metrics (positive predictive value, sensitivity, specificity, and F1 score). An item-level qualitative analysis of misclassifications under the best performing configuration was also completed to identify the source and nature of coding discrepancies.Results: Agreement between ParAlg-generated and human-annotated codes was high, although the orthographic-lexical configuration outperformed phone- mic-only (weighted-average F1 scores of.78 and.87, respectively). A qualitative analysis of the orthographic-lexical configuration revealed a mix of human- and ParAlg-related misclassifications, the former of which were related primarily to phonological similarity judgments whereas the latter were due to semantic simi- larity assignment.Conclusions: ParAlg is an accurate and efficient alternative to manual scoring of paraphasias, particularly when lexical responses are orthographically tran- scribed. With further development, it has the potential to be a useful software application for anomia assessment. Supplemental Material: https://doi.org/10.23641/asha.22087763