GAIN Working Papers: Rasch Analysis of the Behavioral Complexity Scale

Purpose. The purpose of this report is to provide a brief psychometric analysis of the Behavioral Complexity Scale (BCS) using the Rasch measurement model. The BCS consists of 33 past-year yes/no items related to externalizing disorders, including attention deficit, hyperactivity/impulsivity, and conduct disorder. Methods. Data were analyzed on 7,435 persons who presented for substance problem screening. Rasch analysis included an examination of: person and item reliabilities; construct validity including item and person fit statistics; and differential item functioning (DIF) across subgroups. DIF analysis allowed us to determine if the relative item estimates (i.e., item difficulty estimates) remained invariant across subgroups of persons. Results. The BCS performs well as a measure of the construct of behavioral complexity. Both items and scales form the theoretically expected hierarchies with a person internal consistency reliability of .87 and an item reliability of 1.00. The persons' responses generally conformed to the expectations of the Rasch model. Of the 33 items in the BCS, significant DIF (i.e., > .5 SD = .78 logits) occurred in 4 items for males vs. females, 5 items for youth vs. adults, 10 items for race when using African American as the reference group, and 9 items for primary substances when using alcohol as the referent. No items misfit using the criterion of .75-1.33 MNSQ for both infit and outfit. Conclusion. The results suggest the possibility of gender bias. It seems counter-intuitive that females would more readily endorse having committed rape. We suggest that this item should be reworded to make clear that it refers to rape (e.g., It is possible that the item may be misinterpreted whereby some females may be thinking more in terms of seduction). Other DIF analyses did not discern obvious unbalance that could result in biased measurement, but they did reveal some issues that might be of theoretical interest. Citation. Conrad, K. J., Conrad, K. M., Dennis, M. L., & Riley, B.B. (2008). GAIN working papers: Rasch analysis of the Behavioral Complexity Scale. Chicago, IL: Chestnut Health Systems. Retrieved from http://www.chestnut.org/li/gain/psychometric_reports/gain_working_papers_rasch_analysis_of_the_beha vioral_complexity_scale.pdf